This week’s EconTalk is an interview with Morten Jerven on the poor quality of data coming out of Africa. Here’s an excerpt:
Russ: And these are data that are being used–they are used for all kinds of reasons. They are used to try to determine how much aid to give; they are measures of internal value to the country. But they are often used by economists in the development field to assess whether certain policies are working or not working or at least whether certain countries are making progress or not making progress. And when you say they differ–these estimates vary by time and space, meaning across country and even within a country across time–that’s very disturbing for the people who are running sophisticated statistical models trying to assess the effectiveness of various policies. Right?
Jerven: That’s right. Let me just talk through a couple of examples or debates where GDP statistics are particularly important. One is, for instance, whether a country is ranked as poor or a middle-income poor as a country, according to the World Bank. If it is ranked as a poor country, such as Tanzania and Kenya are, then it’s eligible for concessional lending through the International Development Association (IDA), the concessional arm of the World Bank. If it is a middle-income country, it is not eligible for that kind of lending. And to take another example, it was in Ghana, when they re-did their GDP estimates, they recently found out that their economy was almost double the size of what they previously thought and previously published. So that suddenly the Ghanaian economy was ranked as a middle-income country and was no longer eligible for concessional lending. Whereas other countries which have not updated their GDP statistics maybe are–we would hesitate to compare Ghana with Tanzania or Nigeria or Kenya today, and particularly it makes a mockery of those kind of rankings when we recently see, and how I describe in the book, how vulnerable these statistics are. But there are other, as you refer to as more sophisticated econometric analyses using these data. And I think perhaps the most famous debate for those who have been interested in African economic development for some time is that about whether structural adjustment programs or rather, the liberalization programs, which were implemented in sub-Saharan economics almost without exceptions from the 1980s, 1990s; and the big debate was whether this was supposed to spur growth, to make growth recover. And the big debates have always been to try to compare strong reformers to modest reformers, and then try to tease out an average GDP growth effect. Now, when we know how big the underlying availability of these data series are, we know there is enough error in there to make these kind of analyses completely–well, not trustworthy. Russ: I hear you soften it. Completely meaningless is what you meant to say. But ‘not trustworthy’ is very polite; I like that. Guest: Yeah. And the third example, which is one that I think is resurfacing as one of the, I think most important questions on how the poorest in the world are faring at the moment, and that is trying to get to what we refer to as elasticities between these measures. So, to what extent is recent growth causing a reduction in poverty, for instance. And when we look at papers written on that trying to calculate these in relationships between recent GDP growth and recent reductions or increases in poverty, these models are unfortunately way more sophisticated than the underlying data bases allow.
Russ: And so basically you are trying to tease out the effect of liberalization on poverty. And you are saying this chain of causation–ideally, liberalization leads to higher growth, which should lead to less poverty. But your two data sets you are looking at, the two observations on GDP and on poverty, you really don’t know what you are getting.
Jerven: That’s right. And there is also–I think–in the book, I try to suggest one suggestion about talking about data as valid versus data as reliable. ‘Valid,’ talking about the GDP measure, would be the question whether the GDP estimate is correct. Does it capture the real economy 100%? Now we know that a GDP measure of the U.S. economy, the Germany economy, the Norwegian economy, will never be correct. It will always be a little bit off. Some data–there will be some cheating, there will be some data which are questionable. But we know we are more or less within bounds, off a couple of percentage here and there. And so that would be the question of validity. As we’ve seen from recent events in Ghana, and also forthcoming events in Nigeria, the validity question is really huge in sub-Saharan Africa. We are talking about plus-minus 50 to 100% on GDP levels. This would maybe not be a problem if you were interested in change, as we were talking about: what one type of change has a causal effect on another, such as GDP, liberalization, and parity. The problem is if you have that the validity of the measure changes through time. So that would be if you equated this with your bathroom scale at home–it wouldn’t be such a big problem if your personal scale was off a pound or two, if you were basically just interested in measuring yourself on a weekly basis to see if you are gaining or losing. The problem that comes in is that of reliability, and that is if someone changes your scale in the middle of the night. And therefore you have a scale that shows an error in a different direction. And there you will have different problems talking about time series or changes over time. Another problem is that validity still remains with us even if the data was reliable, in that if you started comparing your own weight with that of the neighbor, who uses a different scale, then it would still be very different to determine who is the heaviest or lightest.
