Determining What to Study - Evaluation Methods in Biomedical Informatics - page 62

evaluation questions that frequently arise will be provided and the dilem-
mas that investigators face in the design and execution of evaluation studies
will be examined.
Scenario 1: A Healthcare Problem or Opportunity That
Seems Amenable to an Information or Communication
Resource Has Been Identified, But There Is a Need to
Define the Problem in More Detail
The emphasis here is clearly on understanding the nature and cause of a
problem, and particularly whether the problem is due to poor information
collection, management, processing, analysis, or communication. An infor-
mation resource is unlikely to help if the cause lies elsewhere. It is likely
that careful listening to those who have identified or “own” the problem
will be necessary, as will some observation of the problem when it occurs,
interviews with those who are affected, and assessment of the frequency
and severity of the problem and its consequences to patients, professionals,
students, organizations, and others. The emphasis will be on assessing needs
as perceived across a range of constituencies, studying potential resource
users in the field, assessing user skills, knowledge, decisions, or actions, as
well as work processes, costs, team function, or organizational productivity.
If there are existing information resources in the setting, studies of their use
and the quality of data they hold can provide further valuable insights into
the nature of the problem and reason it occurs. If no information problem
is revealed by a thorough needs assessment, there is probably no need for
a new information resource, irrespective of how appealing the notion may
seem from a technical point of view.
Typical Evaluation Questions
This evaluation scenario typically raises some specific questions that can be
addressed using a variety of evaluation methods. These questions include:
• What is the problem, why does it matter, and how much effort is the orga-
nization likely to devote to resolving it?
• What is the history of the problem, and has anyone ever tackled it before?
How, and with what outcome?
• Where, when, and how frequently does the problem occur? What are the
consequences for staff, other people, and the organization?
• Is the problem independent of other problems, or is it linked to, or even
a symptom of, other problems somewhere else?
• What are all the factors leading to the problem, and how much might
improvements in information handling ameliorate it?