A Controlled Experiment on the Process Used by Developers During Internet-Scale Code Search - Finding Source Code on the Web for Remix and Reuse - page 72

r 2 Terms Queries Clickthrough Time

p-value per query per session rate spent

Queries 0.27 1.00

04 ∗

0 .

Clickthrough 0.02 − 0.02 1.00

1.00 1.00

Time 0.15 0.46 0.28 1.00

001 ∗ 03 ∗ 1.00 0 . 0 . 1.00

P@10 0.02 − 0.10 0.44 0.03

001 ∗ 1.00
∗ The difference is statistically significant at the p < 0.05 level

1.00 1.00 0 .

Table 4.10: Table of correlation coefficients and p-values

factor cause both to rise together. The “mere exposure effect” tells us that we tend
to prefer items that we are more familiar with [ 18 ]. The increased exposure could
be greater frequency or greater total time. In this context, participants may have
been more likely to consider an item relevant if they had clicked through on it previ-
ously. But we also know that participants clicked on a higher proportion of relevant
matches than irrelevant ones, which suggests a causal relationship in the opposite
direction. A clear explanation will require further study.

4.6 Discussion

We compare user behavior when searching for code on the Web with other kinds of
search.

4.6.1 Comparison with General Search

User behavior when searching for code on the Web shared a mix of characteristics
with user behavior in other kinds of search. A prior study by Jansen and Pooch
[ 10 ] summarized characteristics of user behavior when using Web search engines,
traditional information retrieval systems, and online public access catalogs (OPAC).
An example of a traditional information retrieval system would be the INSPEC
database of scholarly articles in engineering, physics, and other fields. Historically,
these databases were available only in libraries and designed to be used primarily
by librarians. OPAC are more commonly known as online library catalogs and the
designed to be used primarily by patrons. We reproduce their table here and add
our data.