Information Technology Reference
In-Depth Information
pretty quickly. Also, all the on-screen keyboards were significantly slower than
the control condition, which was a real keyboard.
4.5.3 Issues to Consider When Measuring Learnability
Two of the key issues to address when measuring learnability are (1) what
should be considered a trial and (2) how many trials to include.
WHAT IS A TRIAL?
In some situations learning is continuous. This means that the user is interact-
ing with the product fairly continuously without any significant breaks in time.
Memory is much less a factor in this situation. Learning is more about devel-
oping and modifying different strategies to complete a set of tasks. The whole
concept of trials does not make much sense for continuous learning. What do
you do in this situation? One approach is to take your measurements at speci-
fied time intervals. For example, you may need to take measurements every 5
minutes, 15 minutes, or every hour. In one usability study we conducted, we
wanted to evaluate the learnability of a new suite of applications that would be
used many times every day. We started by bringing the participants into the lab
for their first exposure to the applications and their initial tasks. They then went
back to their regular jobs and began using the applications to do their normal
work. We brought them back into the lab 1 month later and had them perform
basically the same tasks again (with minor changes in details) while we took the
same performance measures. Finally, we brought them back one more time after
another month and repeated the procedure. In this way, we were able to look at
learnability over a 2-month period.
NUMBER OF TRIALS
How many trials should you plan for? Obviously there must be at least two, but
in most cases there should be at least three or four. Sometimes it's difficult to
predict where in the sequence of trials the most learning will take place or even if
it will take place. In this situation, you should err on the side of more trials than
you think you might need to reach stable performance.
4.6 SUMMARY
Performance metrics are powerful tools used to evaluate the usability of any
product. They are the cornerstone of usability and can inform key decisions,
such as whether a new product is ready to launch. Performance metrics are
always based on user behavior rather than what they say. There are five general
types of performance metrics.
1.
Task success metrics are used when you are interested in whether users
are able to complete tasks using the product. Sometimes you might
only be interested in whether a user is successful or not based on a strict
set of criteria (binary success). Other times you might be interested in
defining different levels of success based on the degree of completion,
Search WWH ::




Custom Search