Determining What to Study - Evaluation Methods in Biomedical Informatics - page 80

Appendix A
Areas of Potential Information Resource Impact on
Health Care, Care Providers, and Organizations
Database
Frequency of data loss, breaches of data confidentiality, downtime and its
consequences, speed of response (e.g., transaction rates or time per trans-
action) when database is in routine use.
Data Coding/Translation Process
Accuracy of coded data following clinical input; problems when coding data
(e.g., percentage of data items that users wished to enter that were suc-
cessfully coded, number of new codes added by users).
Data Retrieval Process
Ease of searching, which search methods are used; time to formulate a
search; user satisfaction with searches.
Data retrieval: completeness, time taken for data retrieval; degradation
with number of cases stored.
Any Data Input Process
Ease of data entry, usage rate, subjective ease of data entry.
Objective accuracy of data entry, time taken, number of actions (e.g., key
presses) taken to enter data items, number of errors correctly detected by
the resource; variation among users, repeatability with same user; learning
effect.
Speech Input System
Ease of use, accuracy in a real environment, number of repetitions needed,
deliberate use of restricted subsets of vocabulary, speed of use, speaker
invariance, resistance to background noise, directionality, frequency that
users enter data or commands via alternate means, percentage of time used
via the telephone.
Knowledge Resource (e.g., as a Full-Text Database, Part
of an Advice Generator)
Users' perceptions of coverage, detail, ease of reading output, speed, how
much is current, ease of using index, finding synonyms.