Evaluation (Validation and Verification) - Medical Device Design: Innovation from Concept to Market - page 238

difficult and expensive to collect (e.g., MRI scans) so this has to be thought through with
great care so as not to miss something. This section should also estimate the number of
subjects required. This is where you need advice from a statistician.
9.6.2.1 Study Types
There are two main study disciplines: prospective and retrospective. A prospective study
starts with a plan and works from a blank page - it is forward looking and is controlled.
A prospective study is the “gold standard” and your marketing staff will love you forever if
you obtain marketing data using this methodology.
A retrospective study looks backward in time, has no plan, and has no real hypothesis; its
sole aim is to look backwards and identify any trends or averages. In real life retrospective
studies are frowned upon as they have not been controlled…however in some cases this is
all a company has so they have to make do with it. However even retrospective studies are
bound by the usual ethical protocols as one is accessing private data for use other than clinical
treatment. Commonly, registrars (interns) tend to do this type of study as a part of their
training and it is normal to go to clinical conferences and be bombarded with presentations,
from newly qualified surgeons, that are clearly retrospective in nature: take them with a pinch
of salt as the data is tainted by the lack of control.
In common practice, there are three main types of prospective study:
- Open: In an open study one is not comparing one device with another; one is
only examining one device. This may be to obtain an average time to set up, or an
assessment of usability. But the data is not for comparison (except against historical
norms, which is not a good idea as there is no control). This is the sort of study
one sees for toothpaste, etc. where a statistic is stated: “ 84% of users said it was
wonderful and would recommend it to a friend ,” or “ 88/90 patients returned to work .”
The trouble with this type of study is that for all we know 99% of existing users think
the same for the existing device, or 90/90 patients return to work with the existing
device…I think you can see the problem. You will need to persuade clinicians,
procurement, medical staff, etc. that yours is the best device to buy; they will always
come back with a question concerning comparison.
- Blind: A blind study is designed to take out the influence of the subject. This is
normally achieved by splitting the study into two subject groups; the first group is
treated using your new device, the second group (called the control group , see Table
9.10 ) is treated using normal practice. It is important that the second group is treated
using normal practice for two reasons: firstly you can compare your results directly;
and secondly your control group is not being disadvantaged - they are still receiving the
best possible treatment. In no circumstances does this mean the use of a placebo. 3
3 A placebo device is one that has no effect at all. Would you be willing to have this treatment?