Performing Subjectivist Studies in the Qualitative Traditions Responsive to Users - Evaluation Methods in Biomedical Informatics - page 289

would do manually, but the software keeps track of the assignments so that
later the segments assigned to each code can be retrieved using keyword
searches. This feature is especially helpful when writing study results. By
reviewing all of the text pieces associated with a particular code, it is easier
to find relevant material to quote.
Programs that provide code-and-retrieve capabilities include HyperQual,
NUD*IST, QUALPRO, and Ethnograph. For example, one interviewee in
Ash's CPOE studies said of an EMR: “It's timely, legible, accurate, com-
prehensive information then, so I think it beats the heck out of the paper
chart, and the one big drawback then and still to this day is that it takes
longer for the doctors I think to go back and type it.” This one sentence
could be coded “benefits,” “drawbacks,” and “time issues.” If the researcher
later wanted to find all text that related to perceptions of the time it takes
to use an EMR and used the software to search on the word time , this
snippet would be found.
As coding progresses, terms can be grouped into higher level patterns
and examined for relationships to build themes and develop theory. Pro-
grams that are especially useful for looking at relations among themes are
NUD*IST, ATLAS, HyperRESEARCH, QCA, and AQUAD. The products
that allow sharing of codes among different researchers are most desirable
for team research. Within this class of software, graphics capabilities for
building conceptual networks are helpful, though not essential. Programs
with these graphics capabilities include Inspiration, Decision Maker,
ATLAS, MECA, and NVIVO. This family of software is also useful for
version control, so members of the team do not chaotically edit each other's
work. For a more complete discussion, see Crabtree and Miller. 26
The NET members decide to analyze data using the “editing style”
described by Crabtree and Miller, and to manage the large amount of data
to be collected by multiple researchers, they select an appropriate software
package. They hold regular analysis meetings to review the coding that each
individual has done and then they agree on “team” codes, patterns, and
themes by consensus. The patterns and themes evolve until all of the data have
been analyzed and a final consensus is reached about what should be
reported.
Presenting Evaluation Results and Criteria for
Evaluating the Quality of Qualitative Inquiry
As a rule, at least at the present time, most people who read evaluation and
research reports in biomedical informatics are more familiar with objec-
tivist methods than with subjectivist, qualitative methods. For this reason,
the presentation of qualitative reports must be undertaken with special
care, clarity, and explanation.