Causality, Impartiality and Evidence-Based Policy - Mechanism and Causality in Biology and Economics - page 213

A regulatory decision should be impartial, and if we are going to ground it on
inconclusive evidence, we need to make sure that nobody exploits such uncertainty
in their own interest.
We have seen that the use of randomisation requires expert judgement, so the
mechanical objectivity of RCTs is mere appearance: we need a subjective (judge-
ment-based) assessment of its actual implementation in order to decide about the
external validity of the trial. Nonetheless, randomisation provides a warrant of the
impartiality of a clinical trial at the crucial stage of allocating treatments - not
beyond that. Such warrant contributes to the credibility of the experimental out-
come: we may question its external validity, but at least we can presume it is
unbiased - at least more than unconstrained expert judgement.
As regulatory tools, RCTs have proven to be most successful at the FDA, where
they are part of a system in four phases: the first two provide causal background
knowledge for the trial, and the last one - post-marketing surveillance - controls for
possible lacks of external validity. The number of label revisions, on the one hand,
and market withdrawals, on the other, signals the levels of uncertainty with which
the FDA is dealing. Randomisation, together with other means (such as blinding),
has contributed to the impartiality of such uncertain regulatory decisions, making
them more acceptable to the American public. However, it seems as though belief
in the regulatory system has weakened today, and a debate has been started on how
to strengthen the fairness of the FDA regulatory process.
If we are going to adopt RFEs as a public policy tool, we will probably need to
work on two fronts. In Sect. 5 , we have argued that randomisation needs to be
successfully hidden from experimental subjects in order to be a warrant of impar-
tiality in field trials, since we are not dealing with the biases of the researchers
alone, but also with the preferences of the experimental subjects. On the other hand,
as Cartwright has argued, and the example of the FDA seems to illustrate, a
randomised trial per se does not warrant the external validity of its conclusions.
We need to keep a record of the fallibility of the conclusions of field trials in order
to measure the degree of uncertainty we are dealing with.
If we follow the institutional paradigm of the FDA, the question is how to
integrate RFEs into an institutional system that makes their results credible. As of
today, there is no clear answer as to which sort of institution should this be (Duflo
and Kremer 2005 ). Government-sponsored programmes are rare because it is
difficult to attain the high level of political consensus required for a successful
implementation. Without this consensus, RFEs can easily fall prey to the sort of
manipulations described in the previous section, in which each party will try to
make the experiment support its views. Non-governmental organisations (NGOs)
are more active, because they are interested in finding the most efficient way of
spending their (usually scarce) resources and they are comparatively free to choose
where and how they distribute them. However, NGOs create their own biases: the
culture of the organisation implementing the assessment (e.g. the motivation of its
employees) may impact on the participants' reaction in a way difficult to replicate
in further extensions of the programme.