Framework for the Review of Research Programs of the National Institute for Occupational Safety and Health - Agriculture Forestry and Fishing Research at NIOSH - page 249

Assessment
Only a qualitative assessment of product development, usefulness, and impact
is required at this point in the EC report. Some thought should be given to the
relative value of intermediate outcomes, and the FC recommends applying the well-
accepted hierarchy-of-controls model. The discussion could include comments on
how widely products have been used or programs implemented. The qualitative
discussion should be specific as to the various products developed by the program
and the extent of their use by specific entities (industry, labor, government, and
so on) for specific purposes. Whether the products have resulted in changes in
the workplace or in the reduction of risk factors should be discussed. The recog-
nition accorded to the program or the facilities by its peers (such as recognition
as a “center of excellence” by national and international communities) should be
considered in the assessment. A program to be highly ranked should have a high
level of performance in most of the relevant questions in this section. Whether the
impact was caused by NIOSH alone or in combination with external agents should
also be considered in the evaluation. An aspect of the evaluation can be whether
the impact would have probably occurred without NIOSH's efforts.

III.B.8. Review of End Outcomes
(Box F in Figure 2)
End outcomes are defined by measures of health and safety and of impact on
process and programs. The FC recognizes that a major challenge in assessing the
causal relationship between NIOSH research and specific occupational health and
safety outcomes is that NIOSH does not have direct responsibility or authority for
implementing its research findings in the workplace. Furthermore, the benefits of
NIOSH research program outputs can be realized, potential, or limited to knowl-
edge gain. For example, negative studies contribute to the knowledge base and the
generation of important new knowledge is a recognized form of outcome, in the
absence of measurable impacts.
Outcome impact depends on there being a “receptor” for research results,
including regulatory agencies, consensus and professional organizations, and em-
ployers. The ECs should consider questions related to the various stages that lead
to outputs, such as

1. Did NIOSH research identify a gap in protection or a means of reduc-

tion of risk?

2. Did NIOSH convey that information to potential users in a usable

form?

3. Was the research applied?

4. Did the results work?