Framework for the Review of Research Programs of the National Institute for Occupational Safety and Health - Agriculture Forestry and Fishing Research at NIOSH - page 244

Assessment
For this part of the assessment, the EC will provide a qualitative assessment
discussing relevance and quality. This evaluation must include consideration of
the external factors identified in Section III.A that constrain choices of research
projects. The EC will consider the appropriateness of resource allocations with
respect to issues' importance and the extent to which the issue is being addressed.
A highly relevant and high-quality program would be comprehensive, address
high-priority needs, produce high-quality results, be highly collaborative, and be
of value to stakeholders. Programs may be progressively less relevant or of lower
quality as those key elements are not up to the mark or are missing. The discussion
should cover those aspects in sufficient detail to arrive at a qualitative assessment
of the activities. Assessment of the transfer activities must include considerations
of program planning, coherence, quality, and impact.
III.B.6. Review of Outputs
(Box D in Figure 2)
As shown in Figure 1, research inputs and activities lead to outputs. An output
is a direct product of a NIOSH research program that is logically related to the
achievement of desirable and intended outcomes. Outputs are created for research-
ers, practitioners, intermediaries, and end-users, such as consumers. Outputs can be
in the form of publications in peer-reviewed journals, recommendations, reports,
Web-site content, workshops and presentations, databases, educational materials,
scales and methods, new technologies, patents, technical assistance, and so on.
Outputs of NIOSH's extramurally funded activities should also be considered.
Examples of major outputs are provided in Table 3.
Depending on the intended audience, outputs may be tailored to communicate
information most effectively to increase the likelihood of comprehension, knowl-
edge, attitude formation, and behavioral intent. The extent of use of formative
evaluation data (data gathered prior to communication for the purpose of improv-
ing the likelihood of the intended effects) or intended user feedback in the design
of the output can be considered an indicator of output quality.
In addition to outputs themselves, many related indicators of the production,
reference to, and utility of outputs can be conceptualized and made operational.
Examples include the extent of collaboration with other organizations in the
determination of research agendas, the conduct of research, the dissemination of
research results, and interorganizational involvement in the production of outputs.
Coauthorship is a measure of the centrality of NIOSH researchers in the broader
research community.