Proposing and Communicating the Results of Evaluation Studies: Ethical, Legal, and Regulatory Issues - Evaluation Methods in Biomedical Informatics - page 345

N - 1. There is no simple solution to this problem when writing a proposal.
The best approach is to describe the dependencies, how the decision about
how to proceed will be made, and possibly describe in some detail the plan
for stage N under what the investigator considers the most likely outcome
of stage N - 1. Some proposal writers consider the existence of a cascade
problem to indicate the boundaries of what they will define as one study.
If the outcome of stage N depends critically on stage N - 1, stage N is
considered to be a different study and is described in a separate proposal
that is written later, after the work on the previous stage has started and
progressed.

Refereeing Evaluation Studies

After a study proposal is submitted to a funding body, it is usually refer-
eed—that is, reviewed for merit by one or more individuals experienced in
evaluation design and methods. It is therefore useful for those writing a pro-
posal for an evaluation study also to understand the refereeing process. In
addition, once an investigator has succeeded in obtaining funding for
several research and evaluation projects, funding organizations are quite
likely to send that individual study proposals to a referee. We therefore
discuss here briefly how one goes about reviewing a proposed study sub-
mitted for funding. Many of these concepts also apply in general to a com-
pleted study that has been submitted for formal presentation or publication.
Many funding organizations or journals provide referees with a checklist of
review criteria they would like addressed, which obviously take precedence
over the generic advice that follows. Some generic criteria from the U.S.
National Institutes of Health are listed in Table 12.2.
In general, the questions that referees can ask themselves, when referee-
ing an evaluation proposal, include the following:

• Is there a study question and specific aims, and are they clearly formu-
lated? Often there is more than one question per study.

TABLE 12.2. Some generic review criteria for study proposals, from the U.S. National
Institutes of Health.
Criterion

Questions for referee/reviewer

Significance Does it address an important problem? How will science or clinical

practice be advanced?

Approach Is the conceptual or clinical framework sound? Are potential problems

discussed?

Innovation Are the aims, concepts, methods, and outcomes novel? Do they challenge

paradigms?

Investigator Does the principal investigator or team have appropriate training and

experience?

Environment Does the study benefit from this scientific/clinical environment?