The obesity “epidemic”

by Russ Roberts on November 30, 2011

in Data, Health

A number of articles have purported to show that obesity and other social problems spread through social networks. These articles, many by Christakis and Fowler, have appeared in the New England Journal of Medicine and other top medical journals. I find such results strange–they generally don’t pass the sniff test for me. The causal mechanism is usually inferred to exist from the statistical significance of the results.

Russell Lyons, a mathematician, has now published a paper in the journal Statistics, Politics, and Policy that finds that the work by Christakis and Fowler is not so good. (HT: Pietro Poggi-Corradini). Here is the summary:

We begin by summarizing the major problems with C&F’s studies:
1. The data are not available to others.
2. The unavailable data are sparse for friendships.
3. The models used to analyze the sparse data contradict the data and the conclusions.
4. The method used to estimate the dubious models does not apply.
5. The statistical significance tests from the questionable estimates do not show
the proposed differences.
6. The wrongly proposed differences do not distinguish among homophily, environment, and induction.
7. Associations at a distance are better explained by homophily than by induction.

(Homophily is selection bias, that “people tend to associate with others like themselves.)

Or in another words, bad paper, meaningless results. It’s not an easy article to follow (and neither was the original work by Christakis and Fowler.) The point on statistical significance is pretty clear though and pretty deadly.

Lyons had a little trouble getting his article published:

We first submitted our critique to the New Engl. J. Med., but it was rejected
without peer review. The journal declined to give a reason when asked. We next submitted to BMJ, but it was again rejected without peer review. This journal did, however, volunteer that “We decided your paper was probably better placed in a more specialist journal.” It is interesting to note that the same issue of BMJ that published Fowler and Christakis (2008a) also published the critique Cohen-Cole and Fletcher (2008a). The cover of that issue, in fact, was devoted to those two articles. In contrast to BMJ’s decision, the general-interest online newsmagazine Slate published an article by Johns (2010) on our critique the same month we submitted our paper. An delightful coda is that a few months later, BMJ published an editorial by Schriger and Altman (2010) called “Inadequate post-publication review of medical research”.

After these rejections by the New Engl. J. Med. and BMJ, we approached
three top journals who did not publish any of C&F’s studies, JAMA, Lancet, and Proc. Natl. Acad. Sci.. All were uninterested in our critique since they do not publish critiques of articles they did not originally publish. The section of J. Pers. Soc. Psychol. that published Cacioppo et al. (2009) does not publish critiques even of papers they have published, unless accompanied by new data.

Following on this educational venture, we submitted to a statistics journal
that specializes in reviews, Stat. Sci. Five months later they had 3 referee reports. The first two recommended publication after revisions (e.g., “an important critique” and “well worth publishing”), while the third, though agreeing with our critiques, said that C&F’s work was insufficiently important to warrant publication of a critique in Stat. Sci. Two months after getting these reports, the editor made his decision: rejection, allowing for resubmission if we made the tone more neutral and changed the focus, perhaps to “editorial decision making standards in medical journals”, as suggested by the third referee.

Methodological journals abound, but their cautions and recommendations
are largely ignored (Blalock, 1989). Indeed, “in a process well documented by
Blalock and Duncan, positivist sociology, like so many other professions, has tended to become immune to the recognition of flaws in its work” (Baldus, 1990). Given the above considerations, it may help to have a journal specifically devoted to critiques. This would not only allow others to know more about which studies are trustworthy, but could also have the salutary effect of encouraging researchers to pay extra attention to their methods lest they be publicly critiqued.

Be Sociable, Share!

Comments

comments

30 comments    Share Share    Print    Email

{ 30 comments }

Josh S November 30, 2011 at 3:35 pm

I thought obesity spread through cheeseburgers via a hand-to-mouth transmission mechanism.

Jon Murphy November 30, 2011 at 3:41 pm

Josh, don’t be ridiculous. Obesity is caused by George Bush and his oil company buddies. They hate America, after all.[/sarcasm]

Methinks1776 November 30, 2011 at 5:36 pm

It’s obviously caused by capitalism and freedom.

What with the producers’ unrestrained ability to cater to the peasants’ primitive penchant for abundant toothsome eats, coupled with the peasants’ fetish for choice, it’s a wonder we survived as a species for so long without the guiding hand of the American Congress.

Greg Webb November 30, 2011 at 9:19 pm

Wait a minute! We wouldn’t have unregulated capitalism if it hadn’t been for that evil Ronald Reagan repealing all the good work that Franlin Roosevelt did in rewriting the Constitution. I’m sure Halliberton and Blackwater are to blame too…though I can’t tell you how.

Mr. Leftwinger December 1, 2011 at 8:08 pm

I’m sure Halliberton and Blackwater are to blame too…though I can’t tell you how.

No need to. The accusation by itself is enough.

I’ll believe.

Josh S December 1, 2011 at 8:08 am

Either way, after my wedding, I’m pretty sure my wife is going to restrict my cheeseburger intake.

mp2c December 3, 2011 at 11:27 am

Great irony here. The post was non-political and dealt quite a bit with people (mainly Journal editors and researchers lacking statistical skill) seeing only what they want to see. The comments come in completely off topic and are stereotyped bizarre fantasies of what a mythological lefty would come up with. Unable to think in other patterns?

mp2c December 3, 2011 at 11:28 am

haha….maybe sharing fries are the social network aspect.

Larry Wasserman November 30, 2011 at 3:46 pm

A couple of my colleagues have an article
casting doubt on this stuff, as well.
The article is here:

http://arxiv.org/abs/1004.4704

–Larry Wasserman

ThomasL December 1, 2011 at 1:00 pm

Thanks for the link.

jdcllns November 30, 2011 at 3:46 pm

Capitalism! Bad, bad capitalism!

Michael November 30, 2011 at 3:55 pm

I’m not even sure I can follow the basic premise. How does one take seriously the idea that weight spreads through social networks? It isn’t communicable…

vikingvista November 30, 2011 at 7:33 pm

Actually, some have theorized a virus.

Methinks1776 November 30, 2011 at 7:59 pm

Does this virus happen to look anything like a bucket of fried chicken so large your cat could use it as a bathtub when you’ve finished eating the contents?

vikingvista November 30, 2011 at 8:48 pm

Not really, but it has been observed to order a large meat lover’s pizza with a size of wings and a liter of Coke from PappaJohnsOnline.com. Here’s a SEM from the J of Microbiology demonstrating this mechanism:

http://tinyurl.com/7bmkmux

Greg Webb November 30, 2011 at 9:20 pm

Don’t forget that special sauce of melted butter and garlic!!!!!

Methinks1776 November 30, 2011 at 9:23 pm

Finally! A visual of one of the Cafe Hayek trolls!

mp2c December 3, 2011 at 11:34 am

I can see how it could happen (which is very different from believing that it does). A big part of socializing is sharing meals. Meal choices matter. If you’re coworkers or friends tend to order apps or desert instead of just a main course, then you are more likely to. If you’re an active person, many of your friends will also be. Beer drinkings tend to have other beer drinkers in their networks etc. None of this indicates that there is major causality (i.e. predisposition towards a life style would have you select people in that life style rather than your network influencing you).

Mark T November 30, 2011 at 5:16 pm

The stuff about the journals’ biases is equally important, thanks for publicizing that.

vikingvista November 30, 2011 at 6:14 pm

You are skeptical about a peer-reviewed published article? You must be a holocaust denier. I have an idea for the title of this proposed journal of critques–how about “Holocaust Denier’s Weekly”? Or “Creationist Monthly”? Or simply “Antiscience”. If they wanted to really discover the truth, they’d have a vote of say, 10,000 Congressionally-approved experts on each article, and just publish the poll results.

What do you think, Muirde?

brotio November 30, 2011 at 8:10 pm

What do you think, Muirde?

The next time Yasafi thinks will be the first time.

Justin P November 30, 2011 at 10:02 pm

+1

Krishnan November 30, 2011 at 8:51 pm

Talk about the “seen” and the “unseen” – We find out that the NEJM – that GREAT, GREAT Journal for Medical Science has quacks – they cannot stand the fact that someone critiques what they have published. I wonder how many such papers they have refused to publish – OR how many papers that contradict what they they have gets into print. Calls into question their judgement.

kyle8 November 30, 2011 at 9:49 pm

Wait! Are you trying to say that Sociology is mostly a bunch of hogwash? Well, knock me over with a feather!

indianajim November 30, 2011 at 10:20 pm

Hmmmm… Our government planners subsidize food production and consumption and then wonder: Why are people fat?????

I’m sure that the planners/subsidy recipient nexus is NOT the social network effect that Christakis and Fowler studied.

kyle8 December 1, 2011 at 7:23 am

Not only do they subsidize food, but they subsidize exactly the wrong types of food like Carbohydrate rich grains and corn sweetener.

William December 1, 2011 at 5:07 pm

Penn & Tell covered this on their TV Show Bullsh*t. They criticized the Body Mass Index and addressed some other myths relating to weight.

The episode can be viewed here: http://www.viddler.com/explore/britomart/videos/1/

mp2c December 3, 2011 at 11:35 am

I wonder if BMJ rejected it because they had already published a critique of the original article?

ezra abrams December 4, 2011 at 10:28 pm

Dr Lyons begins his abstract as follows
“The chronic widespread misuse of statistics is usually inadvertent, not
intentional.”
And who, exactly , should get the blame for this ?
It seems to me that the blame goes, largely, to those responsible for the teaching of statistics.

Also, in my experience, major journals like JAMA and BMJ typically publish criticisms of papers only as letters to the editor. The blog post doesn’t explicitly state if the paper was submitted to JAMA as a paper or as a letter to the editor

Rob McMillin December 9, 2011 at 2:09 pm

I have thought for a long time that if some of these pseudo-scientific frauds (the A in AGW is the biggest one that comes to mind) are ever to be unhorsed, it will occur not in the journals germane to those fields but in the statistical journals.

Previous post:

Next post: