Information Technology Reference
In-Depth Information
6.2.3 When to Collect Self-Reported Data
During a usability study, you might collect self-reported data in the form of ver-
batim comments from a think-aloud protocol while the participants are inter-
acting with the product. Two additional times when you might want to probe
more explicitly for self-reported data are immediately after each task (post-task
ratings) and at the end of the entire session (poststudy ratings). Poststudy rat-
ings tend to be the more common, but both have advantages. Quick ratings
immediately after each task can help pinpoint tasks and parts of the interface
that are particularly problematic. More in-depth ratings and open-ended ques-
tions at the end of the session can provide an effective overall evaluation after
the participant has had a chance to interact with the product more fully.
6.2.4 How to Collect Ratings
Logistically, three techniques can be used to collect self-reported data in a usabil-
ity test: answer questions or provide ratings orally, record responses on a paper
form, or provide responses using some type of online tool. Each technique has
its advantages and disadvantages. Having the participant provide responses
orally is the easiest method from the participant's perspective, but, of course, it
means that an observer needs to record the responses, and may introduce some
bias as participants sometimes feel uncomfortable verbally stating poor ratings.
This works best for a single, quick rating after each task.
Paper forms and online forms are suitable both for quick ratings and for lon-
ger surveys. Paper forms may be easier to create than online, but they involve
manual entry of data, including the potential for errors in interpreting hand-
writing. Online forms are getting easier to create, as evidenced by the number
of web-based questionnaire tools available, and participants are getting more
accustomed to using them. One technique that works well is to have a laptop
computer or perhaps tablet computer with the online questionnaire next to the
participant's computer in the usability lab. The participant can then refer to the
application or website easily while completing the online survey.
ONLINE SURVEY TOOLS
Many tools are available for creating and administering surveys via the web. Doing a
search on “online survey tools” turns up a pretty extensive list. Some of them include
Google Docs' Forms, Qualtrics.com, SnapSurveys.com, SurveyGizmo.com,
SurveyMonkey.com, SurveyShare.com, and Zoomerang.com. Most of these tools support
a variety of question types, including rating scales, check boxes, drop-down lists, grids,
and open-ended questions. These tools generally have some type of free trial or other
limited-functionality subscription that lets you try out the service for free.
Search WWH ::




Custom Search