Information Technology Reference
In-Depth Information
The between-participants variable was type of search interfaces (with or with-
out auto-suggest). The within-participants variable was inclusion of the domain
vocabulary (with or without domain vocabulary). In this section, we describe
aspects of the evaluation method, beginning with the experimental system.
4.1 System
In order to focus on the auto-suggest interface and control system conditions, the
feature of faceted search was deactivated and only the keyword search was made
available in the school document portal site. The portal site was customized
into four ways in accordance with the 2x2 factorial design. The baseline system
provided only the keyword search function without the auto-suggest interface,
where the domain vocabulary was not included in the indexing terms and the
weight of indexing terms was calculated without considering the domain vocab-
ulary. That is, the effect of
Z
i
is
effective. On the other hand, the full-featured system provided the auto-suggest
interface, and the domain vocabulary was considered for the indexing terms and
their weighting function. In the other two systems, either of the auto-suggest
feature or the domain vocabulary was taken into account in addition to the
baseline system.
X
i
in the equation (1) is nullified while
Y
i
and
4.2 Method
Participants.
A total of 32 undergraduate students were recruited from Faculty
of Informatics, Kansai University. The participants were without prior experi-
ence of using the school document portal site, and paid to participate in the
experiment.
Apparatus.
We tested all the participants in a lab setting on the campus, using
Mozilla Firefox 3 browser on Windows Vista workstations with 19-inch monitors
set at 1280 by 1024 pixels in 24-bit color. The system ran on a server located in
the campus, and data was recorded in the server log.
Experimental Tasks.
Some search tasks are exploratory and requires searchers
to gather information on a subject to allow them to perform some action [15].
In contrast to such an open-ended search, this portal is primarily used for seek-
ing particular pieces of information or school documents in a target language.
Therefore, a type of tasks to be investigated here is not exploratory search but
known-item search [17]. The known-item search is to seek for a particular in-
formation item under the searchers' anticipation that such an item exists, and
sometimes referred to as navigational queries in Web site search [18].
In this study, we suppose a situation where the school document portal is
used to find out and download document files of a desired topic in a target
language. Participants are asked to assume they are a teacher in an elementary
school, and a situation looking for a Portuguese document that meets specified
needs. Table 2 shows the descriptions of the experimental tasks presented to the
participants.
