Information Technology Reference
In-Depth Information
Table 4.
Mean values in the case of two-minute search time (SD in parentheses)
NAS
AS
NDV
DV
NDV
DV
Number of queries
3.16 (1.34)
2.16 (0.72)
1.66 (0.58)
1.41 (0.44)
Precision
0.05 (0.04)
0.13 (0.09)
0.13 (0.05)
0.19 (0.07)
Recall
0.48 (0.20)
0.66 (0.22)
0.95 (0.09)
0.72 (0.16)
NAS = no auto-suggest, AS = with auto-suggest,
NDV = no domain vocabulary, DV = with domain vocabulary.
5.2 Number of Queries
The number of queries formulated in each task is shown in Table 4. As will be
noted from the table, the number of queries was reduced to less than 2.0 by
the auto-suggest interface, regardless of the inclusion of the domain vocabulary.
A two-way ANOVA was conducted to determine if there was a significant dif-
ference in the average number of queries between the search interface type and
the inclusion of the domain vocabulary. Significant main effects were found for
the search interface type [
F
(1
,
30) = 27
.
380
,p<
0
.
001] and the inclusion of the
domain vocabulary [
01]. There was no significant inter-
action between the factors. In each case, the number of queries was smaller when
either the auto-suggest interface or the domain vocabulary was adopted.
It is reported that the mean number of queries
7
in a user session is 1.73 (with
a median of 1) for Web site search in a local government site [10]. This value
is much lower than the mean value 4.86 (with a median of 8) observed in the
case of a general-purpose search engine [9]. Since the type of search dealt in
this study is not a general-purpose search but a Web site search, the number of
queries improved by the auto-suggest seems to be a moderate level with regard
to the case of the local government site. Needless to say, the number of queries
alone would not be a good indicator of the search effectiveness. However, if we
consider the high success rate as well as the reduced number of queries, it is
reasonable to acknowledge the effect of the auto-suggest interface.
F
(1
,
30) = 7
.
797
,p<
0
.
5.3 Precision and Recall
Table 4 shows precision and recall averaged across queries formulated by each
participant within the two-minute search time. A two-way ANOVA was per-
formed to examine differences in the precision and recall. ANOVA for recall
revealed that there was a significant interaction [
001]
between the search interface type and the inclusion of the domain vocabulary.
As for the precision, significant main effects were found for the search interface
type [
F
(1
,
30) = 19
.
486
,p<
0
.
F
(1
,
30) = 26
.
348
,p<
0
.
001] and the inclusion of the domain vocabulary
[
F
(1
,
30) = 10
.
942
,p <
0
.
005]. In each case, the precision was improved, and
7
Not as unique queries but queries including repeat queries.
