Sunday, June 22, 2008

Not a week goes by...

until some some study comes along for a news source to confuse correlation with causation. I just wrote about emos and now it's mobile phones that are bad for teenagers:

...teens who made more than 15 phone calls and sent more than 15 text messages a day, slept poorly and had more careless lifestyles compared to those who made less than five of each per day.

Conducted by Sahlgren's Academy in Sweden using 21 teenagers...

Note it is only on 21 subjects. Not exactly a large sample size.

Now, as I said with my emo post, I wouldn't be surprised if it is the excessive use of the phone that is the cause of the lack of sleep. But, it could just as easily be the lack of sleep means they have nothing better to do than text their friends... You need to rule out this hypothesis before you know the direction of the cause. Otherwise you could be making the False Cause; Correlation Error. Perhaps the authors did this, but if so, the article does not make this clear.

What it does go on to say, after the quote above:
...the study found teenagers who made at least 15 calls and messages a day spent more time on computers and drank more caffeinated drinks and alcohol, had more irregular sleeping hours and found it more difficult to wake up and were more tired before midday.
And as you can see, we have uncontrolled variables. If we want to establish the mobile phone use as a causal mechanism for poor sleep, we'd need to rule out the other potential causes, e.g., the caffeine and booze. Perhaps they are addicted to caffeine (a drug) which keeps them up, which leads them to text and myspace, which keeps them mentally stimulated, which leads to more coffee or Red Bull, etc. A vicious circle or sorts, and another hypothesis.

To see the effect of the phone on its own, you'd need to control for these other variables. A simple experiment to set up. Perhaps the "natural" experiment has been done. Is there evidence of an increase in teen sleep deprivation (above and beyond better diagnoses) over the last 10-15 years?

Moreover, the article then goes on to relate this study to an unrelated one:
The second study, conducted by Fred Danner at the University of Kentucky on 882 Year 9 students, found teenagers who slept less than eight hours a night got worse grades and had a higher level of emotional disturbance and risk of ADHD.

The unstated claim is lack of sleep leads to worse grades. If this is the direction then I won't be surprised. But surely you have to get some students who were doing poorly to sleep more and see if their grades improve?

Note that by linking these two studies together, we now have the implication that using mobile phones leads to poor grades

I'll tell you what I would bet on. When questioned at length, I'd bet the authors of these studies would qualify these claims along the exact lines I have. The number one thing they'd say is more research needs to be done. In order to form any real position on claims in a field of current research it's best to take studies like this as interesting, but not conclusive. Wait and see where we are at in another ten years.

A problem with non-science journalists reporting science, is they don't often put qualifiers like this in the story. This can to lead to claims of Exaggerated Conflict from those wanting to dispute scientific claims. Think about climate science, or the reporting of stories on health and nutrition. Ben Goldacre writes about this very point in his Bad Science blog.