Causality, Impartiality and Evidence-Based Policy - Mechanism and Causality in Biology and Economics - page 208

run in the United States between 1960 and 1995 (Orr 1999 ), with more or less
convincing results.
In the last decade, there has been an explosion of interest in RFEs among
development economists. Several programmes for improving health or education,
different microfinance and governance schemes have been tested in a number of
developing countries. A success story is the PROGRESA programme implemented
in Mexico in 1998. PROGRESA aimed at improving school performance through a
system of direct transfers conditional on family income, school attendance and
preventive health measurements. The amount of the allocation, received directly by
the mothers, was calculated to match the salary of a teenager. In order to test the
effects of PROGRESA (and with a view to secure its continuation if there was a
change in government), a team at the Ministry chose 506 villages, implementing
PROGRESA in a randomly selected half of them. The data showed an increase in
teenager enrolment in secondary education significantly higher in the experimental
group, with concomitant improvements in the community health. The experiment
was considered convincing enough to ground the extension of the scheme to more
than 30 countries.
The boom of RFEs in development economics may owe something to their
costs: in developing countries, the costs for running these programmes are signifi-
cantly lower than, say, in the United States, and non-governmental organisations
can implement them in a quick and efficient manner. But there is also a sense of
political opportunity among these social experimentalists. A leading one, Esther
Duflo, puts it as follows: just as RCTs brought about a revolution in medicine, RFEs
can do the same for the assessment of our education and health policies in fighting
poverty (Duflo 2010 , p. 17).
Nonetheless, Duflo acknowledges that RFEs can involve many methodological
pitfalls. Randomisation is a case in point. Field experimentalists in economics
expect it to provide a solid foundation for causal analysis, and we have already
discussed Cartwright's criticism of this idea. In this section we discuss further
whether we can take RFEs in development economics to be impartial. More
precisely, our question is whether randomisation is as credible warrant of
impartiality in field trials development in economics as it is in medical RCTs.
We think not.
Let us present our case by drawing on an analysis due to James Heckman. In
1992 , Heckman published a seminal paper containing 'most of the standard
objections' against randomised experiments in the social sciences. Heckman
focused on the non-comparative evaluation of social policy programmes, where
randomisation simply decided who would join them (without allocating the rest to a
control group). Heckman claimed that even if randomisation allows the experi-
menters to reduce selection biases, it may produce a different bias. Specifically,
experimental subjects might behave differently if joining the programme did not
require 'a lottery'. Randomisation can thus interfere with the decision patterns
(the causes of action) presupposed in the programme under evaluation.