Fifteen leading economists, including three Nobel Prize winners, are fed up with randomized control trials (RCTs) determining the future of poverty alleviation. So last month, they banded together to speak out against what they call the “aid effectiveness” craze.
They are pushing back a growing trend to use the same kind of trials that are used in medicine, ones that assess “treatment” groups against control groups to ensure that aid programs are actually working. Ensuring efficacy seems like a good idea, but the economists worry it has a darker side.
“It narrows our focus down to micro-interventions at a local level that yield results that can be observed in the short term. It tends to ignore the broader macroeconomic, political, and institutional drivers of impoverishment and underdevelopment,” the economists wrote in a letter. “Aid projects might yield satisfying micro-results, but they generally do little to change the systems that produce the problems in the first place. What we need instead is to tackle the real root causes of poverty, inequality, and climate change.”
Among the letter’s signatories was Barbara Harriss-White, a professor at Oxford University who specializes in political economy, South Asia, and rural and local development. BRIGHT Magazine’s Sarika Bansal caught up with her recently to talk about the false promise of micro-interventions and randomized control trials when it comes to development, and where she believes the industry should move instead.
BRIGHT: I was impressed with the letter you co-signed, along with 15 prominent economists. What prompted this action?
Barbara Harriss-White: The letter was borne out of a general sense of frustration with the growing use of randomized control trials (RCTs) to determine development policy in the context of widespread poverty.
The signatories of the letter argue that the problem with this thinking is that it does not take into account macro-economy, history, culture, and politics. Such is its diversity that even locally for many development questions, there can be no social “controls.”
RCTs are essentially technocratic — and since the intervener usually stands outside the society s/he aims to intervene in, the way it is practiced is also often at odds with concept of social development as democratic deliberation. By their nature, RCTs also tend to delegitimate other ways of knowing: practical experiential knowledge, triangulation, observational, longitudinal, and historical approaches. They are also very costly and divert resources from other ways of knowing.
BRIGHT: I think one of the reasons the aid industry veers towards RCTs, or micro-interventions, is because it’s easier to measure progress. How do you think we can best measure change in large systemic efforts?
BHW: You can only argue that micro-interventions are successful if you believe you can hold human societies constant, as in the method of lab experimentation.
If an experiment shows that when you increase rural teachers’ pay they teach “better” (measured by outcomes) than in the control where pay is not raised and outcomes are poorer, then how do you increase rural teachers’ pay? The education budget is constrained — why? There is widespread tax evasion. How is reducing tax evasion to be researched through RCTs?
A further point is that the entire discipline of anthropology, for example, is dedicated to the richness and varied nature of human society. For decades we have known that tribal society differs from caste society. Now we know that, as in villages, the social structure of one town may differ from that of the next and so do their politics. RCT experiments will have to make simplifying assumptions in their research design. The factors simplified out may well stymie the outcome of the policy that might result from the experiment.
On measuring systemic change, please consider how you would measure the effects of the Russian revolution, or of late 20th century neoliberalism. My most hopeful reply is that it is too early to measure large-scale systemic efforts to change society.
BRIGHT: Can you say more about the varied nature of human society, and what this implies for long-term policy change when it comes to development and aid?
BHW: We now know that different kinds of human underdevelopment map onto income poverty in different ways. So we know there is no magic bullet for human deprivations. We also know that the same policy is implemented differently in different societies and regions. So we cannot avoid the challenges of policy specificity.
Chief among these challenges is that being specific about specific forms of deprivation requires good local understanding of society and the actually existing state. Bina Fernandez in her 2012 book for example showed how often a good policy idea, even if it has been costed (which many policy suggestions aren’t), will trickle through the bureaucracy with late release of funds, theft for private uses, top-slicing the budget for other public uses, bribery, incompetence and laziness, redesigning the queues of eligible people so the ineligible enter the queue and much more besides.
These practices are balanced differently in different regions. Since we also know that cleaning up bureaucratic behavior requires cleaning up party political funding and criminal economic activity, the high probability that in the short term such distortions will occur in aid-driven funding is a fact that needs factoring in and not simply hand-wringing. What is the opposition to a well-justified policy? How can that opposition be worked around, destroyed, or bought? What are the underlying social and political conditions that will make a policy work as intended (and costed)? These three questions should be mainstreamed in all aid agencies, all business schools, and all public policy teaching.
BRIGHT: The letter talked about how, as an example, “Water purification tablets are too little in the face of droughts induced by climate change.” What if any role can micro-interventions play in ending poverty?
BHW: In the case of water purification tablets, which are a micro-intervention to bring clean drinking water to populations struggling with climate change, a great deal of radical transformative development is needed first to adapt to droughts, then to attempt to mitigate them in future and to invest in clean water.
Micro-interventions can certainly act as dressings, but cannot heal the wound. Tablets cannot solve the underlying problem causing scarce and polluted water.
BRIGHT: I enjoyed your paper on markets and human development, particularly this line: “The way markets are conceived affects how the relation between markets and human development is understood…. There are no ‘free markets.’” What lessons do you think the aid industry can take from this observation?
BHW: Several books have come out this year arguing that belief in free markets is like a religion. Belief that free markets will cure distorted allocations of resources flies in face of the fact that — as Nobel Laureate Amartya Sen once said — markets are consistent with any distribution of demand, including societies where some have no resources and no demand at all. If people have no money to buy health, food, or education — the big three of human development — there won’t be a demand for them. Given that, very few markets are competitive and efficient.
The objective of business is, ultimately, to maximize profit (if a family–run business) or shareholder value (if a public company). If businesses fail in this objective, they go bankrupt. The market is further distorted in the developing world, which largely consists of oligopolies and masses of tiny firms dependent in various ways on these oligopolies.
Even if human development is not a classic “public good” (which is defined as being non-excludable and non-rivalrous), human development has to be a project led by non-market action. Human development is a public good in the sense that society has decided politically that it be provided though public action.
BRIGHT: Part of the reason the aid industry is so flawed, I believe, is that their budgets are tucked away from the rest of a government’s priorities. What actions do you think donor countries can make throughout their governments to reflect your proposed reframing of the aid industry?
BHW: Would you have the aid budget part of the Foreign Ministry because that is the practical alternative? The U.K.’s Department for International Development has tried to ring fence their aid budget, but it’s leaking out to other ministries.
If aid remains an independent part of government, the questions I was posing in answer to your previous question are relevant here. Always listen respectfully to progressive forces in the poorest countries who are willing and able to share their analyses and ideas. (But how do you find them?)
If I had a magic wand, the biggest ‘top-down’ question for reframing aid that I would wish for is to satisfy the urgent need for a low-carbon and employment-intensive transition.
This a big challenge for several reasons: the so-called advanced countries are not leading the way (and may not know how, or be able to do this given the power of fossil fuel interests); less advanced countries need resources to act independently (and it is not clear they want to lead the way); development policy is mired in aspiration and lacking in hard facts about technologies, their costs, social and environmental achievements and trade-offs, their protection and finance. The world needs a user-friendly, multi-language portal with basic information about all frontier technologies in all sectors which are polluting the biosphere. At present: far too little and far too late.
BRIGHT: What do you hope will be the outcome of the letter?
BHW: Replies like yours! I don’t have much faith in the corporate media these days. And well-argued pressure from people like you on aid and foreign ministries.
