Causality, Impartiality and Evidence-Based Policy - Mechanism and Causality in Biology and Economics - page 207

for its contribution to causal analysis but rather for preventing anyone to exploit this
uncertainty about causation for her own benefit. The same justification applies to
other standard features of RCTs such as masking treatments or having pre-
established decision rules for the interpretation of the results (as in significance
testing): they provide a priori warrants for the impartiality of the trial.
The research protocol in RCTs constrains expert judgement at various critical
points in the generation and interpretation of clinical evidence during a trial. In this
sense, pharmaceutical regulation is based as much on the impartiality of its eviden-
tial base as on the accuracy of its causal conclusions. Perhaps there are other
sources of evidence whose external validity is as good as RCTs, but it is an open
question if they can be as impartial as the latter. Impartiality is crucial for public
policy, and it seems a defensible decision to adopt RCTs instead of mere expert
judgement for regulatory purposes: at least, the former provide certain warrants of
impartiality.
An obvious objection, of course, is that RCTs are not actually impartial. There is
evidence showing, for instance, a sponsor bias: industry-funded trials published are
more likely to support the experimental therapy than the standard alternative,
despite their good methodological quality. We know that RCTs do not control
every possible source of bias, e.g. the research protocol does not impose any
constraint on the research question that trials should address, and there is no
obvious way to decide which one should it be. But rather than an objection against
RCTs, it should be a general caveat about every possible source of clinical evi-
dence: the easier it is to manipulate the method, the less we should rely on the
evidence it produces for regulatory purposes.
Summing up, despite the problems with their external validity, regulatory
RCTs have been reasonably efficient in keeping the American pharmaceutical
market clear of unsafe or inefficacious compounds. Moreover, despite all the
label revisions, the American public has considered the FDA a reliable regulator
(Carpenter 2010 ), and we contend that this is because RCTs provided a warrant of
impartiality for their decision, despite the inherent uncertainty of phase III trials.
If we had full information about the effects of a therapy, impartiality would be
warranted by default. However, short of that, we need to make sure that a regulatory
decision is fair despite their inherent imperfection.
5 The Impartiality of Randomised Field Evaluations
The assessment of public policy programmes through large-scale randomised field
evaluations (RFEs) is already several decades old (the 1968 New Jersey negative
income tax experiment is often considered to be a pioneering example). Usually the
interventions assessed deal with one or another aspect of the welfare of large
populations, and testing them is expensive, though the cost of the actual implemen-
tation of the programme would be significantly more so. Around 200 RFEs were