Autism and TV

by Russ Roberts on February 27, 2007

in Data

Here’s a very interesting article in the WSJ (HT: Steve Saletta) on the use of what is called  instrumental variables in econometrics to argue that television watching causes autism. It seems like a goofy idea and it may be. The more interesting issue for me is the use of sophisticated statistical techniques to try and observe cause and effect. Too much of the time, for my taste, instrumental variables are used to awe and amaze rather than to ferret out the truth. And too many economists have become enamored of technique for technique’s sake.

The paper discussed in the Journal is a perfect example and the reporter, Mark Whitehouse does a superb job explaining the econometrics for a non-expert:

Prof. Waldman, a recognized expert in the field of
applied microeconomics, doesn’t pretend to be an authority on autism.
He became engrossed in the subject in the fall of 2003, when his
2-year-old son, David, was identified as having an autism-spectrum
disorder. Hoping to eliminate any potential triggers, Prof. Waldman
supplemented the recommended therapy with a sharp reduction in
television watching. His son had started watching more TV in the summer
before the diagnosis, after a baby sister was born.

Prof. Waldman says his son improved within six months
and today has fully recovered — a surprising result, given that autism
is typically a lifetime affliction. "When I saw the rapid progress,
which was certainly not what anyone had been predicting, I became very
curious as to whether television watching might have played a role in
the onset of the disorder," he says. He tried to get medical
researchers interested in the idea, to no avail.

In late 2004, he decided to look into the subject
himself, ultimately putting together a research team with Cornell
health economist Sean Nicholson and Nodir Adilov, a professor of
economics at Indiana University-Purdue University in Fort Wayne.

In principle, the best way to figure out whether
television triggers autism would be to do what medical researchers do:
randomly select a group of susceptible babies at birth to refrain from
television, then compare their autism rate to a similar control group
that watched normal amounts of TV. If the abstaining group proved less
likely to develop autism, that would point to TV as a culprit.

Economists usually have neither the money nor the
access to children needed to perform that kind of experiment. More
broadly, randomized trials seldom lend themselves to studying economic
questions, particularly the more traditional ones. It would be unfair
to randomly subject some people to a higher tax rate just to see how it
affects their spending.

Instead, economists look for instruments — natural
forces or government policies that do the random selection for them.
First developed in the 1920s, the technique helps them separate cause
and effect. Establishing whether A causes B can be difficult, because
often it could go either way. If television watching were shown to be
unusually prevalent among autistic children, it could mean either that
television makes them autistic or that something about being autistic
makes them more interested in TV.

The ideal instrument is a variable that is correlated
with A but has no direct effect of its own on B. It should also have no
connection to other factors that might cause B. If data in a study
nonetheless show that the instrumental variable is linked to B, it
suggests that A must be contributing to B.

After giving a few examples of previous studies that used instrumental variables, the article gets back to Waldman and autism:

By putting together weather
data and government time-use studies, they found that children tended
to spend more time in front of the television when it rained or snowed.
Precipitation became the group’s instrumental variable, because it
randomly selected some children to watch more TV than others.

The researchers looked at detailed precipitation and
autism data from Washington, Oregon and California — states where rain
and snowfall tend to vary a lot. They found that children who grew up
during periods of unusually high precipitation proved more likely to be
diagnosed with autism. A second instrument for TV-watching, the
percentage of households that subscribe to cable, produced a similar
result. Prof. Waldman’s group concluded that TV-watching could be a
cause of autism.

Criticism quickly arose, illustrating some of the
perils of the economists’ approach. For one, instruments are often too
blunt. As Prof. Waldman concedes, precipitation could be linked to a
lot of factors other than TV-watching — such as household mold — that
could be imagined to trigger autism. At best, his data reflect the
effect of television on those children who changed their habits because
of rain or snow, not on those who did it for other reasons such as a
desire to watch educational shows.

On the surface, the whole thing sounds vaguely plausible. Watching TV is a brain activity. It’s imaginable that watching TV affects your brain and could, at a crucial developmental stage, cause the brain to do various things, both good and bad.

But then you start to think about it a little more. They didn’t show a relationship between TV and autism. They showed a relationship between rainfall and autism. Then they made the leap that more rainfall means more TV. But more rainfall means a thousand other things, too. The "theory" that encouraged the study was based on ONE observation, that Waldman’s kid watched less TV and got better. But surely there were thousands of things that were different in the Waldman household over those six months.

Then there is the problem of defining autism. There are different degrees of autism. Were they all lumped together? How precise was the relationship between rainfall and autism? What was the confidence interval? Did the definition of autism change over the period? The diagnosis of autism is much more common today than in the past, either because of environmental factors or more plausibly, a higher awareness of the phenomenon by both doctors and patients and a widening of the definition of what is considered autistic. Maybe an increase in rainfall just happened to coincide with an expansion of the definition. Surely the increase in cable TV subscriptions is correlated with that expansion as well. And is there really so much variation in snowfall and rainfall in Oregon, California and Washington states? Why were those states chosen? If anything, people in Portland and Seattle are used to lots of rain and snowfall and probably don’t respond by watching TV a lot when it’s raining. Did the authors control for these possible effects?

When I hear these kind of speculative findings (and they are very common in the profession these days) I always want to ask the researchers how confident they really are in their findings and have they really discovered anything we didn’t already know.

In the case of the autism study, I’d ask the following: does the econometric firepower brought to bear on the problem add anything more to our knowledge than the anecdotal story that Michael Waldman’s son got better after watching less TV? I’m not convinced. But here’s what I really want to say to the authors of the study.

You found a correlation between rainfall and autism that might indicate a correlation between TV and autism. Let’s take a thousand kids with autism who currently watch a lot of TV. Let’s do a real experiment where we remove their access to television. How much are you willing to bet that there will be a statistically significant reduction in autism? A thousand dollars? Ten thousand dollars? Ten dollars? Put your money where your econometric mouth is. You found something. Do you really believe it?

Be Sociable, Share!

Comments

comments

26 comments    Share Share    Print    Email

{ 13 comments }

Bill Conerly February 27, 2007 at 12:16 pm

The Waldman report is exciting in that it may point the way for more research, but by itself it's scary. Take the entire universe of possible instruments: rainfall in Oregon, sidewalk temperatures in France, earthquakes in Japan. If you can't find something to correlate with your subject, you ain't really trying. Spurious correlation is even easier to find in data that don't have multiple cycles.

George McCallum February 27, 2007 at 12:29 pm

Another question that doesn't seem to have been addressed was: what activity was substituted when TV watching was forbidden?
Could it have been that the parents began (again) to spend more time with the older child?
Was TV simply a convenient fantasy world that the child escaped into when it felt (suddenly) neglected because of the birth of the second child?

Tim Worstall February 27, 2007 at 1:11 pm

Sigh. The rise in autism cases happened to coincide with the spread of cable TV in the early 80s, correct.
That's also the time that the diagnosis of autism was changed, from what we might terms "classic autism" to the whole spectrum, including such things as Aspergers.
Possibly the best researcher (well, personal opinion there) is Simon Baron Cohen of Cambridge University.
The idea that there's a spectrum of brain types (from "male" to "female" with autism being an extreme form of the male) with men and women having a probability, but not a certainty, of having the brain type that corresponds to their gender.
Assortative mating means that more male types are marrying, leading to more expressions of the extreme male type.
A decent introduction here:
http://www.eqsq.com/theory.php
(Disclosure, yes I am paid to write for that site. It's still interesting though.)

Mickey Klein February 27, 2007 at 2:23 pm

Just another epedemic of diagnosis…

Matt C. February 27, 2007 at 3:11 pm

I read an article a few months back that stated that there may be an increase in, incorrect diagnosis of Autism. Similar to what happened several years ago with ADD/ADHD.

I also wonder if there is a correlation of TV watching to mental development of children. So maybe they are less expressive because they become wrapped up in television, they don't learn how to express themselves properly because of the decrease in interaction with parents and other play mates. Just a few questions I have.

Chris O'Leary February 27, 2007 at 3:57 pm

Russ,

Thanks for bringing this up.

I am just now coming to grips with the fact that I have Asperger's Syndrome (a flavor of autism that makes me very good at somethings and very bad at others), and this is very interesting.

The only question/concern that I have is that he may be confusing cause and effect. Is TV watching the cause of the problem or just a symptom of it? I could see how it could make things worse.

mcewen February 27, 2007 at 4:01 pm

It also makes for an interesting read if in conjunction with the findings about 'older fathers' being 'responsible' for the 'epidemic' of autism.
Sounds like we're moving back to a genetic / parental / refrigerator mother mindset.
Lets hope that the pendulum swings back the other way soon.
Cheers

pinus February 27, 2007 at 11:00 pm

Instrumental variables?

I downloaded the article from NBER. I admit I haven't read the whole thing carefully (67 pages), but I did a thorough skimming of the methodology and the tables.

What they is not instrumental variables, at least in the sense econometricians know it from 2SLS.

First, they use data from the American time use survey together with the weather data for the days when the surveys took place to establish that more precipitation => more television. I have no problem with that, they seem to consider quite reasonable covariates, so that the result seem to be quite robust. They do OLS and tobit, get similar results in terms of significance, fair enough.

Then they say, OK, we know that bad weather causes more TV watching, let's feed the precipitation variable directly into the equation for autism as an explanatory variable. But the data for autism is completely different – they regress the autism rates in particular cohorts in different counties on the West coast against the average precipitation rates in these counties some years back (at the times when the children were babies).

They again find precipitation highly significant.

But that's not instrumental variables approach! That's rather using a proxy for TV watching, or an indicator, and moreover a bad one, since TV watching depended also on other significant factors.

Instrumental variable approach would mean projecting TV watching on precipitation and other factors, and then feeding the projections into the equation for autism. But they cannot do this, since they work with two different datasets.

They are running into potentially serious problems with spurious regressions etc. Namely, A causing B and A causing C does not imply C causing B (the relationships between A and B and between A and C can be caused by different components of A). Further, contrary to IV approach, there is no way how to verify the quality of the proxy/indicator that they are using.

Although the idea might be interesting, I am skeptical about the estimation.

Mike February 28, 2007 at 8:22 am

I think Russ's post points to a serious problem in graduate school economics today. I have an army of acquaintences trained at MIT and other similar schools who are armed with a natural experimentation weapon and obsess over ways to apply this technique to address very interesting questions.

No doubt, the work is interesting – and the folks I know are extremely bright, but this comes at the expense of teaching basic economics in graduate school. My classmates simply had no basic economic intuition. Preparing for exams with them was just depressing – they were like automatons who understood the finest details of every technique in the book, but could not explain to their siblings what the basic economic concepts we were studying were all about. Any of you who have taken qualifiers using Mas-Collel for your micro course probably have had similar experiences.

David February 28, 2007 at 10:45 am

Yo, the bottom line for me is this- who can I sue for claiming educational benefits of their TV/video programming? Sesame Street? Baby Einstein? Shouldn't Elmo come with a warning lable now? Do I even need to have an autistic child to file a claim?

colson March 2, 2007 at 11:42 am

Ok, but did the study link television or did they even go far enough to determine the type of programming that was being consumed on TV. I still remember that whole Japanese cartoon seizure scare a few years back and could be led to believe that the correlation isn't with TV per se but the actual content being viewed. I believe the link between seizures and TV had to do with certain types of flashing on the screen that was being used at the time.

Vanya June 3, 2007 at 1:24 pm

Everyone's points on econometrics are very good. A lot of what's going on now seems to still be in the realm of academic exercises rather than rigorous, useful research.

But on the point of TV and autism:

It makes sense to me that an autistic child would process TV content differently from a normal child, so that even a very good, educational program, like Blues Clues, that attempts to be somewhat interactive, could go right over an autistic child's head. So while another child would look at the character's faces and pay attention to what they were saying to follow the story, an autistic child might be more fascinated by the colors or other details in the abstract.

It also makes sense to me that TV watching could be bad for an autistic child's development at least in the sense that time spent watching TV is time not spent being forced into interacting with other people, which is what autistic kids need a lot of practice with.

So it bothers me that so many people (in general, not necessarily on this blog) dismiss the possibility of a link between the two outright. Just because the "refrigerator mother" thesis was wrong, doesn't mean that there's nothing a parent can do wrong (or right!) by an autistic child.

Colleen July 20, 2007 at 8:49 pm

I watched my nephew grow up and now he is 2.5 and has obvious PDD. I love my sister, but she swore by Baby Einstein which seems to be to put children in a trance. I mean how stimulating is it to watch puppets that say nothing or just "blah" and objects spin around and around with no sense or meaning? Even if too much TV is not the cause of autisim, I believe that if you put a 10 month old in front of TV and they scream every time you turn it off, you need to take your kid to the doctor. So really TV could be either the "Envrionmental" trigger they speak of or could be a criteria meet for diagnosing children at a very young age. I have read tons of blogs from mothers who talked about how they thought they were educating their children by putting these two dimensional educational videos in front of their children. I do believe that many children to have neurological deficits due to more structural abnormalities of the brain, but a childs brain is a growing brain and I believe is being "programmed" along the way. My sister's child was diagnosed with Autisim just this past week. She's been in denial for about a year now. I just went to visit her and again her child is sitting in front of a Baby Einstein video. Even if television is not the "cause" of autisim, mothers and fathers need to toughen up and deal with the tantrums their children throw when not getting their way and put their childrens precious growing years to more productive use.

Previous post:

Next post: