Databases Reference
In-Depth Information
top of the list. In addition to the extra work, it takes longer to judge the relevance of
a match, because the subject has to navigate to the Web page and read portions of it.
4.5.2 Main Effect of Intention (
H
1
)
The intention of the search affected all the dependent variables. This was the only
independent variable that had an effect across the board. Reference examples had a
positive effect on the values of all variables. Subjects expended more effort - queries
with more terms, sessions with more queries, a higher clickthrough rate, and longer
session duration and were rewarded with better results, i.e. a higher P@10 rate.
The additional time, queries, and terms indicate that the subjects were trying a
wider variety of searches. They were tweaking their queries by adding a term (spe-
cialization), which was often followed by removing a term (generalization.) They
were trying out search keywords and filters to reduce the large number of matches
and to bring the most relevant ones to the top of the results set.
Relevant reference examples were easier to find, likely because the criteria for
judging relevance were more flexible. Overall, subjects had a higher clickthrough
rate with relevant results. So the higher proportion of relevant results when searching
for reference examples led to the higher clickthrough rate.
4.5.3 Main Effect of Search Engine (
H
1
)
The search engine used affected all the dependent variables except for the number
of queries. Google seemed to stand out from the others: subjects used more terms,
judged greater P@10, navigated to more matches, and spent more time.
These differences were likely due to the larger quantity and variety of Web pages
that were available through Google and subjects' previous experience using this
search engine. Since Google indexes the entire Web, more pages were available,
so subjects had to be more specific in their queries by using more terms. The larger
number of pages likely contributed to the higher relevance rate. The variety of pages
likely led to the higher clickthrough rate and time spent, i.e. subjects had to look
more carefully at the matches.
4.5.4 Interaction Effects
We found four statistically significant interaction effects, each with a different com-
bination of independent and dependent variables.
There was an interaction effect on the number of terms in a query under combi-
nations of size and intention. Searches for blocks to be used as reference examples
Search WWH ::
Custom Search