Bad Prediction, Bad Graphics

by Russ Roberts on May 17, 2009

in Data, Stimulus

A number of people have sent me versions of this graph that Greg Mankiw has now noted. I delayed posting on it because the graph, which criticizes the Obama administrations predictive abilities, is so bad:

Stimulus effects.jpg

The graph is supposed to show that unemployment with the stimulus package is the same or maybe worse than what the Obama Adminsitration economists said would happen if the stimulus package didn't pass.

The basic point is right. The unemployment rate has continued to rise rather than peak and decline.

But the triangles that are supposed to represent the current levels of unemployment are incredibly large. Why aren't they points? Or smallish circles. Which part of the triangle represents the measure of unemployment? The midpoint? The top? The right vertex?

Worse, the underlying graph is very hard to read. It is taken from the Romer-Bernstein forecast of the effects of the stimulus plan. But the horizontal axis is bizarre. For 2009, for example, there are two tick marks, Q1 and Q3, first quarter and third quarter. Where are Q2 and Q4 measured for 2009 or any year. Presumably, Q2 is halfway between Q1 and Q3 and Q4 is in the no man's land halfway between 2009's Q3 and 2010's Q1.

But now look where the triangles are located. Because they are so big, they overlap multiple quarters. The base of the April 2009 triangle ranges from Q2 all the way over to Q3. But either way, they're not in the right place. They span Q1 and Q3. But we don't even have all the data for Q2 of 2009. So what is the April triangle doing over Q2 and Q3? Why is a month of unemployment put into a chart that has quarterly measures.

But the basic point is likely to be correct. The underlying Romer-Bernstein chart says that unemployment will be 8% for the second quarter of 2009 if there's a stimulus plan and about 8.5% if it doesn't pass. I don't think we're going to average 8% for April, May and June.

Finally, I don't even know what quarterly unemployment is. I assume it's the average during the quarter.
Be Sociable, Share!

Comments

comments

48 comments    Share Share    Print    Email

{ 24 comments }

Ironman May 17, 2009 at 10:39 am

If you prefer points or smallish circles, perhaps this version of the graph will be more to your liking:

http://www.willisms.com/archives/stimulusfail.png

The flaws with the unmodified original however are something I can't address – I would also presume that the data is for the average percentage of individuals counted as unemployed for each of the indicated quarters.

Ironman May 17, 2009 at 10:41 am
Mike Sproul May 17, 2009 at 11:05 am

Since Mankiw's textbook devotes so much space to the same Keynesian theories that are being used to justify the stimulus package, Mankiw deserves some of the blame for Obama's failed economic policies.

Sonic Charmer May 17, 2009 at 11:40 am

This was, far as I can tell, a homemade plot leveraging off an earlier, more official, plot. The triangle locations are likely to be whatever Excel has as defaults, which (I would wager) are the triangle middles. I don't find the plot particularly hard to read. Nor do I find it particularly problematic that the x-axis grid spacing involves Q1 & Q3 only, especially if that's what the original plot used (which I don't know). What sort of grid-scale accuracy do you demand in your estimated-and-bogus-to-begin-with 'unemployment rate' numbers anyway?

But more to the point, if you have all these problems with the plot and data as presented, rather than write a blog post of your own complaining about it to everyone and no one in particular, why not actually contact the author and just ask for clarification. He has a blog, he's the one who posted it there, so he would have these details for you (or refuse to give them to you, which would be a different story), so why not ask him – as you would in a situation where you saw a graph in an academic journal that needed clarification? I know you know where it came from cuz you linked it. Instead we get this post which basically says nothing at all. Odd.

Mathieu B├ędard May 17, 2009 at 1:00 pm

The triangle locations are likely to be whatever Excel has as defaults

That's definitely not Excel's default theme… I was wondering the same thing though, which software is responsible for those atrocities?

dg lesvic May 17, 2009 at 1:02 pm

Sonic Charmer,

Why was it Robert's or Lesvic's or anyone else's obligation to chase down Mankiw for clarification, when it was his obligation to be clear in the first place, and, obviously, he had no intention of it?

Or was he just that incompetent?

dg lesvic May 17, 2009 at 1:04 pm

By the way, Mankiw has made himself as inaccessible to questioning as he possibly could, long since having shut down the Comments section of his blog.

Russell Roberts May 17, 2009 at 1:30 pm

It's not Mankiw's graph.

I give the source in a link just before the link to his comments.

No one has to contact that source. The data are public and well-known. The problem is fundamental–we don't have 2009 Q2 figures and won't have them until the first Friday in July.

TrUmPiT May 17, 2009 at 2:26 pm

Thank you for taking my advise to use your mathematical skills to make cogent and useful arguments. I think you are on the path of improvement and enlightenment.

My late father once read "The Hite Report" which was a bestseller on the latest findings on female sexuality. He told me that he had a math phobia from a young age, and that he was unable to understand the graphs and charts in the book, so he simply skipped them. Good math skills are useful and fun. Be happy you have them, and can use them profitably in your chosen profession.

dg lesvic May 17, 2009 at 6:18 pm

It is precisely the non-mathematical, non-graphical, non-professorial, and old-fashioned, logical and literary, plain and simple, real economics that distinguishes this site.

Keep up that great work, and let Trumpet go blow his horn somewhere else!

Sonic Charmer May 17, 2009 at 7:20 pm

No one has to contact that source. The data are public and well-known.

The fact that the data are public and well-known argues for less not more potential ambiguity in interpreting things like 'what the triangles mean'. Since you know the data so well already, all you had to do was cross-check the triangle locations against what you know to be the data. You wouldn't have had to write this post at all.

The problem is fundamental–we don't have 2009 Q2 figures and won't have them until the first Friday in July.

That's not a "fundamental" problem. The author has taken the April report and overlaid it on a graph that (you are correct) otherwise plots data biquarterly. Unless you're saying the April release somehow connotes something fundamentally incompatible with what the quarterly releases represent, there's no "problem" here at all.

On clarification your objections are even odder than I'd first kenned.

Sonic Charmer May 17, 2009 at 7:24 pm

P.S. And again, if it's the location of the triangles seeming to be too far right (partially over Q3) that bothers you, that would be the sort of thing that would be a perfectly a question for the author. He may indeed have erred, and if so, and made to realize it, could correct it himself.

dg lesvic May 17, 2009 at 7:27 pm

Sonic Charmer,

I've been in economics for half a century, and I don't know what you're talking about.

Is that how you're going to change the world?

Daniel Kuehn May 18, 2009 at 6:09 am

Ya – a lot of people critiqued this when it came out as being overly rosy. It's really a shame because now the higher than predicted unemployment rates are going to be touted by some as definitive proof that the stimulus package made things worse (ironically taking the initial bad estimate and treating that as a reasonably counterfactual). Then again, if they put it higher people would accuse them of lowering the bar on themselves. It's always easy to armchair quarterback these things. What did you estimate unemployment would be, Russ – I'm curious?

Predicting unemployment six months to a year in advance is like predicting weather six months to a year in advance. The economy and the weather are both a complex system. We shouldn't get so hung up on this graph, which the administration was really obligated to produce in some version or another – and pay more attention to monthly BLS unemployment projections. If those are consistently higher or lower than actual unemployment rates, that's far more disconcerting.

Daniel Kuehn May 18, 2009 at 6:11 am

dg lesvic -
RE: "It is precisely the non-mathematical, non-graphical, non-professorial, and old-fashioned, logical and literary, plain and simple, real economics that distinguishes this site."

This argument you love to make always reminds me of a great Keynes line:

"Economists set themselves too easy, too useless a task if in tempestuous seasons they can only tell us that when the storm is long past the ocean is flat again."

Sometimes you have to get your hands dirty to do something useful, dg, and sometimes that requires math.

Sonic Charmer May 18, 2009 at 6:53 am


I've been in economics for half a century, and I don't know what you're talking about.

Is that how you're going to change the world?

dg lesvic,

I don't know what you're talking about.

dg lesvic May 18, 2009 at 12:54 pm

Since Mankiw has said some very good and worthwhile things, in plain English, it is clear that he is capable of doing so, and, when he doesn't, because he has nothing good and worthwhile to say.

And while those of us who have been around economics for any length of time didn't need Roberts to point that out to us, the undergraduates and newcomers to economics that he instructs may have. So, he did his job, and did it well. Thanx and congratulations for that, whether the fakers like it or not.

And, Faker No 1, you had your chance, and blew it, so crawl back in your hole.

And, Sonic Charmer, what I meant was that, if I don't know what you're talking about, I doubt that very many other people do, and that, if you really want to communicate with others, you'd better find a better way to do it.

Daniel Kuehn May 18, 2009 at 1:43 pm

Haha – am I Faker #1?

Look dg – you reject anything you don't understand as being illegitimate economics because you presume nobody else can understand it. I'd be a little more circumspect about accusing other people of having nothing good and worthwhile to say.

dg lesvic May 18, 2009 at 2:23 pm

You should be.

Daniel Kuehn May 18, 2009 at 3:02 pm

should be….?

dg lesvic May 18, 2009 at 3:12 pm

Should be!

Daniel Kuehn May 18, 2009 at 3:17 pm

Oh, THAT clears it up :-)

dg lesvic May 18, 2009 at 4:38 pm

Sorry I don't have a mathematical formula to make it clearer.

geoff May 21, 2009 at 6:30 pm

But the triangles that are supposed to represent the current levels of unemployment are incredibly large.

Whine, whine, whine. I made large triangles because I wanted them very visible at even a momentary glance (which is all you often get in blogging). The centerpoints are located correctly, but were handplaced on top of the original graph, so as to underscore the point that I was using the Obama teams' actual data.

BTW I don't think anybody, ever, in the history of plotting data, ever, ever considered placing the hot point of the triangle anywhere but the center. So I'm not sure why you were feigning confusion on that issue.

Personally I prefer more rigorous-looking graphs, but I've got to say: this graph was linked by everybody (Instapundit, Hot Air (twice), Michelle Malkin (twice), AoSHQ (twice), PowerLine, The Corner, etc.). My latest plot, which took two orders of magnitude more work and has a very traditional Excel technical presentation, has gone nowhere.

Lesson learned: I think I'll stick to large triangles henceforth.

The underlying graph comes from Obama's own team. I suspect they used Excel07. The abscissa is, as you say, bizarre, but comparing it to historical data led me to believe that the labels on the abscissa are at the beginning of each quarter. That's why the data for the end of March is at the mid-point of the labels for the 1st and 3rd quarters. The point for April then lies 2/3 of the way between the 1st and 3rd quarters.

Any other interpretation would move the markers to the left, which would make the Obama teams' predictions look even worse.

Previous post:

Next post: