Behavioral and Physiological Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 174

One way to analyze these data is to take an average of all the times at which
the particular element was first fixated. Data should be treated as elapsed time,
starting from the initial exposure. The average represents the amount of time
taken to first notice the element, for all of those who did notice it. Of course,
it's possible that some of the participants may not have noticed it all, let alone
within the first 5 seconds. Therefore, you may come up with some misleading
data showing an artificially quick time by not taking all the participants into
account.
REVISITS
Revisits are the number of times that the eye fixates within an AOI, leaves the
AOI, and returns back to fixate within the AOI. Revisits indicate the “stickiness”
of the AOI. Do the users fixate and leave the AOI, never to return, or do they
keepcomingbackwiththeireyes?
HIT RATIO
The hit ratio is very simply the percentage of participants who had at least one
fixation within the AOI. In other words, this is the number of participants who
saw the AOI. In Figure 7.10 , 10 out of 13 participants (or 77%) fixated within
this particular AOI.
7.2.5 Eye-Tracking Analysis Tips
Over the years we have learned a few things about how to analyze eye-track-
ing data. Above all else, we strongly recommend you plan your study care-
fully, as well as taking time to explore the data. It's very easy to draw the wrong
CAN YOU TRUST WHAT PEOPLE SAY THEY SAW IN A
USABILITY TEST?
Albert and Tedesco (2010) ran an experiment in which they used eye tracking to
test whether usability test participants report what they see accurately. In this study,
participants looked at a series of website homepages. After being shown each homepage,
the moderators pointed out a specific element. Half of the participants indicated if they
had looked at specific elements based on three potential answers (did not look at the
element, not sure if they looked at the element, or did look at the element). The other
half of the participants used a five-point scale based on how much time was spent
looking at that element (from “no time at all” up to “a lot of time”). Results showed
that, in general, the eye movements were consistent with what the participants reported
seeing. However, in about 10% of the cases, the participant claimed to have “definitely
seen” an element, which the eye-movement data showed they did not fixate. In the
second group of participants, about 5% of the cases the participants said they “spent a
long time looking at an element,” yet did not have any eye fixations on that element.
Together, these results suggest that participants self-reporting what they looked at during
a usability test are reasonably reliable but certainly not perfect.