Can you relate? Correlation and simple linear regression - Improving the User Experience through Practical Data Analytics - page 208

“OK,” Hans says slowly and cautiously. “And?”
“After folks have used the search for a while, we could ask all the participants to
rank their perception of usefulness with the different search ields, along with their
likelihood to adopt the search engine. Then, I can calculate the correlation coeficient
between the usefulness of the ability to perform a Boolean search and likelihood of
adoption of the search engine, and perform some simple regression.”
Hans hesitates: “I have no idea what you just said, but it sounds feasible. When
can we have the results?”
“Well, it'll take some time to get the test together and screen for the right partici-
pants. After the results come in, we'll do the number crunching. Give me 2 weeks.”
“Ok, but no more than 2 weeks right? Bad news doesn't get any better with time.”
He throws down his inished butt, stomps on it, turns abruptly on his heels and leaves
in a huff.
Again, you're off to the races to get answers for Hans and to alleviate his Bool-
ean angst. Realizing that you need higher sample sizes to make a convincing case
for your results, you decide to go with an unmoderated online test of the current
Behemoth search engine. The e-mail invite goes out to about 300 recruiters who are
regularly searching for candidates. After answering some basic eligibility questions,
they are carefully screened to disqualify any current Behemoth customers; you want
newbies who've never used the engine before.
All the respondents are tasked with inding good candidates for the same three
requisitions: (1) A Java Developer with at least 5 years experience, a bachelor's
degree from MIT, a maximum salary requirement of $95,000 per year, willing to
relocate, who is looking for a full-time position; (2) A Web Designer with skills using
Axure, Photoshop, and Illustrator within 25 miles of San Diego, with an Active Con-
idential Security Clearance; and (3) A Business Analyst who has previously worked
at Oracle, an average length of employment of no less than 1 year, with a resume
dated no earlier than 2013, willing to travel up to 50% of the time.
After completing the tasks of inding candidates for the three positions, the respon-
dents are asked about overall satisfaction with the search engine. In addition, they are
speciically asked to rate their perception of usefulness for each of the ields in the
search engine, on a scale of 1-5, where 1 = not at all useful and 5 = extremely useful.
Table 9.1 shows the 15 speciic search engine components respondents are asked to rate.
At the very end of the survey rating, you insert the moment of truth question: “Imagine
that this search engine is available to you at no cost to ind qualiied candidates using the
candidate databases you currently employ. Rate your likelihood of adopting this candi-
date search engine on a scale of 1-5, where 1 = not at all likely and 5 = extremely likely.”
With everything in place, you launch the online study. After a week, you check
into your online test tool. You're happy to ind 233 responses. However, there were
36 incompletes, and another 17 who you have to disqualify for suspicious looking
activity (mostly in the form of overly quick task completion times). You end up with
180 bona ide test responses. You download the Excel spreadsheet containing the rat-
ing scales of the search engine components.
Time to roll up your sleeves. The irst thing you want to establish, of course, is the
perceived value of Boolean search, and its correlation with likelihood of adoption.