One of my daughters — she majored in Psychology — quotes the following: "There are three types of lies: lies, damn lies, and statistics."
Buried in all the elections coverage was the piece by Associated Press reporting results of a study that linked teenage pregnancies to certain television programs. And it's yet another great example of how research leads people astray.
Here's a quote from the AP story:
The study was released Monday in the November issue of Pediatrics. It involved 2,003 12- to 17-year-old girls and boys nationwide questioned by telephone about their TV viewing habits in 2001.
Teens were re-interviewed twice, the last time in 2004, and asked about pregnancy. Among girls, 58 became pregnant during the follow-up, and among boys, 33 said they had gotten a girl pregnant. Participants were asked how often they watched any of more than 20 TV shows popular among teens at the time or which were found to have lots of sexual content.
The programs included "Sex and the City," "That '70s Show" and "Friends." Pregnancies were twice as common among those who said they watched such shows regularly, compared with teens who said they hardly ever saw them. There were more pregnancies among the oldest teens interviewed, but the rate of pregnancy remained consistent across all age groups among those who watched the racy programs.
So what does that prove? Sounds like it says those programs were bad for the kids. But is that what is really shows?
Here's the real question: Would those same kids have had the same behavior if those shows hadn't existed? I get it that the ones who got pregnant were more likely to report watching those shows. Maybe they were just more honest about it.
The trouble is confusing cause and effect. What do you think the chances are that the same study could have found correlation in lots of other factors: kids who chew gum, kids who read, or kids who don't read, kids considered cool by their classmates, or not, kids with hair color, kids with cars.
And then there's the problem of truth in surveys. Watch what people do, and you get better information than you get by asking them what they think.
This kind of linking between different factors is what makes psychographics work. Link behaviors up, like the likelihood of buying a Volvo related to the likelihood of wearing sandals and eating sushi. Eating sushi doesn't cause people to drive Volvos.
And the bad news, if you start trusting research for business use, is that the dressing up of statistical research can make it seem more real, or more valid. Stay skeptical.
One thought on “The Third Type of Lie”
I read this earlier and didn't question the methodology too much, but now I see I should have. Freakonomics really drove home the fact that correlation does not mean causality. As you point out, you could draw correlations between people who eat sushi and drive Volvos, but one does not cause the other.
You must log in to post a comment.