Can you relate? Correlation and simple linear regression - Improving the User Experience through Practical Data Analytics - page 209

SIDEBAR: GET THOSE SAMPLE SIZES UP! UNMODERATED REMOTE
USABILITY TESTING
Unmoderated remote usability testing is a low-cost and effective technique for collecting task-based
usability data. As a consequence, the technique has experienced great growth in recent years by UX
professionals.
Unmoderated remote usability tests allow you to:
• Collect data easily and eficiently.
• Find the right participants.
• Conduct studies on a limited budget.
• Test users in their natural environment.
• Test internationally without traveling.
• Validate or deine lab-based research.
• Achieve statistical signiicance when comparing designs, identify areas of a Web site that need
improvement, and conduct competitive benchmarking studies.
Those are the advantages, but how does it work? In a nutshell, the participant receives an e-mail
invitation to participate in the study, and clicks on the link to begin. He/she is taken to a Web site
where he/she is asked to complete a series of tasks. During task completion, data are being collected
on a wide range of variables, like task completion rates, time to complete the tasks, pages visited,
data entered, etc. Once inished, the participant is asked a series of questions regarding satisfaction
on several different variables. In our case with Behemoth, the participant is asked to rate the useful-
ness of speciic components of the search engine.
For an outstanding introduction and how-to guide on online usability studies—including plan-
ning, designing, launching, and data analysis—we heartily recommend Beyond the Usability Lab:
Conducting Large-Scale Online User Experience Studies , by Bill Albert, Tom Tullis, and Donna
Tedesco (Morgan Kaufmann, 2010). The topic shows you how to use commercial tools, like User
Zoom and Loop 11, but also offers discount approaches that can yield perfectly good results.
Table 9.1 The 15 Search Engine Components
1. Ability to search by job title
2. Ability to search by years of experience
3. Ability to search by location
4. Ability to search by schools attended
5. Ability to search candidates by date of updated resume
6. Ability to search candidates by level of education
7. Ability to search by skills
8. Ability to search candidates by average length of employment at each company
9. Ability to search candidates by maximum salary
10. Ability to search candidates by job type he/she is looking for: full time, part time,
temporary/contract, per diem, intern
11. Ability to search candidates by companies in which they have worked
12. Ability to search candidates by willingness to travel. (Expressed as “no travel ability
required,” “up to 25%,” “up to 50%,” “up to 75%,” “up to 100%”)
13. Ability to search candidates by willingness to relocate
14. Ability to search candidates by security clearance. (Active Conidential, Inactive Con-
idential, Active Secret, Inactive Secret, Active Top Secret, Inactive Top Secret, Active
Secret/SCI, Inactive Top Secret/SCI)
15. Ability to perform a Boolean search