Causality, Impartiality and Evidence-Based Policy - Mechanism and Causality in Biology and Economics - page 204

that the FDA does not take the external validity of an RCT for granted. A drug trial
is usually divided in four phases. Phase I focuses on finding the appropriate dosage
in a small group of healthy subjects (20-80 patients); toxicity and other pharmaco-
logical properties of the drug are examined. In phase II, between 100 and 200
hundred patients are closely monitored to verify the treatment effects. If treatment
effects are detected, a third phase involving a substantial number of patients begins
in which the drug is compared to the standard treatment. This is usually referred to
as 'the' RCT. If the new drug proves to be at least as good as the existing therapies
and the pharmaceutical authorities approve its commercial use, phase IV starts: the
drug is freely prescribed and sold; adverse effects are monitored and morbidity and
mortality studies are undertaken.
In other words, the FDA, as other regulatory agencies, does not take the external
validity of the RCTs for granted when it approves a new substance. In the post-
market surveillance phase IV, the FDA collects adverse event reports from various
sources and conducts epidemiological studies to assess their relevance, keeping
track of the validity of the results of their trials in the general population. The
authority and resources of the FDA at this stage are disproportionately smaller than
at any previous point in the approval process. And the assignment is large: apart
from monitoring adverse reports, the agency also has to consider issues in labelling,
advertising or the inspection of production and storage facilities, to name but a few.
Hence, one should appraise the figures collected at this fourth phase cum mica salis .
But we think they are relevant in the context of our analysis of the reliability of
RCTs.
Changes in drug labelling constitute a first approximation to the number of
minor or major failures at phase III trials. According to Dan Carpenter ( 2010 ,
p. 212), the FDA has relied on these changes as a cheap regulatory strategy,
given the available resources, as compared with pursuing withdrawal or a change
in advertising and prescribing practices (advertising and prescription are only
lightly regulated in the United States as compared to Europe). As long as the
label records potential safety threats, the FDA can claim that the consumer has
been warned. Each label change requires an application for approval, which creates
a data record. Dan Carpenter has compiled it in the following table, where it
is compared to other product changes for the same periods (Fig. 11.1 ):
Carpenter ( 2010 , p. 623) summarises it as follows: from 1980 to 2000, the
average new molecular entity received five labelling revisions after approval,
about one for every 3 years of marketing after approval. Only one in four drugs
had no labelling revisions at all. The data are obviously too coarse to decide what
went wrong, if anything, in the phase III RCTs. Several explanations are possible:
for instance, the trials might have been too brief to detect adverse effects (e.g.
toxicity or cardiovascular events). In the context of Cartwright's analysis, we may
suspect that the patients' sample might have been unrepresentative of the patients
that finally used the therapy. In a rough sense, the list of prognostic factors and
potential confounders used to define the eligibility criteria was incomplete and
randomisation could not correct this flaw. If we take external validity in an equally
rough sense, Carpenter's data would suggest that Cartwright points out correctly the