Information Technology Reference
In-Depth Information
Table 3.
Average rate of search success (SD in parentheses)
NAS
AS
Time (min)
NDV
DV
NDV
DV
1
0.09 (0.20)
0.28 (0.31)
0.22 (0.31)
0.31 (0.24)
2
0.38 (0.38)
0.63 (0.33)
0.59 (0.40)
0.69 (0.24)
3
0.56 (0.39)
0.84 (0.23)
0.84 (0.23)
0.81 (0.24)
NAS = no auto-suggest, AS = with auto-suggest,
NDV = no domain vocabulary, DV = with domain vocabulary.
was used to determine if there was a significant difference in the success rate
between the search interface type and the inclusion of the domain vocabulary.
There were significant main effects of the inclusion of the domain vocabulary
in the search time limit of 1 minute [
F
(1
,
30) = 4
.
175
,p<
0
.
05] and 2 minutes
[
05]. In each case, the success rate was higher when the
domain vocabulary was included. There was no significant interaction between
the factors in both cases.
On the other hand, when the search time was limited within 3 minutes, a sig-
nificant main effect was found for the search interface type [
F
(1
,
30) = 4
.
291
,p<
0
.
F
(1
,
30) = 4
.
342
,p<
.
0
05], and the success rate was higher when the auto-suggest interface was used.
There was a tendency of interaction [
F
,
.
,
.
<p<
.
10] in
this case. This tendency was revealed due to the considerably low success rate
(0.56) in the baseline system, in which neither the auto-suggest interface nor
the domain vocabulary was provided, while the success rates in the other three
conditions were greater than 0.80.
According to the above results, it was demonstrated that the domain vocab-
ulary contributed to the improvement of the success rate during the first two
minutes. In contrast, the effect of the auto-suggest interface became more in-
fluential when the search time elapsed had exceeded two minutes. The rates of
search success in Table 3, however, consider only the success or failure of the
search. In order to further investigate the effectiveness of the search assist fea-
ture, we focus on the number of queries in each task as well as precision and
recall.
It is important to note here that when the search time limit was three min-
utes, mean plus standard deviation of the success rate went beyond 1.0 or the
maximum value in the three of the four conditions (Table 3). This situation
suggests a ceiling effect that may depress averages below the true values. This
effect might be present due to the relatively decreased complexity of the search
task when performed for more than three minutes. In addition, when the search
time was limited to one minute, the maximum success rate was 0.31. Thus the
one-minute search time is too short to be considered for the assessment of the
search effectiveness. In the following analyses, therefore, we take the case of
the two-minute search time into account.
(1
30) = 3
488
0
05
0
