Information Technology Reference
In-Depth Information
forgetting a topic, but, as we will see, facilitates the process. The way differ-
ent hierarchical subnetworks are formulated to account for the topics of interest
is reminiscent of the work in [21], where the author describes a deterministic
algorithm for generating meta-stable network structures based on multivariate
data. An extensions of this work has been applied in the context of ubiquitous
computing [22].
Overall, with Nootropia, a single multi-topic profile can be theoretically adap-
ted to both short-term variations and occasional more radical interest changes.
In contrast to evolutionary algorithms, a single structure and not a population of
profiles, is adapted through a deterministic process, rather than through prob-
abilistic genetic operations. It remains to be shown experimentally that this is
indeed an effective approach to profile adaptation.
4
Experimental Evaluation
The main goal of this paper is to demonstrate Nootropia's ability to adapt
to a variety of interest changes in a user's multiple interests. For this purpose
we needed an evaluation methodology that reflects the multimodal and time
dependent nature of this problem. Unfortunately no existing evaluation standard
fulfilled our requirement. Even the adaptive filtering track of the well established
Text Retrieval Conference, concentrates on evaluating the ability of a profile,
that represents a single topic category, to adapt to modest changes in the content
of documents about that topic. It does not simulate radical changes in a user's
multiple interests. For the evaluation of Nootropia and other biologically inspired
solutions to IF a more challenging setting is required. After all, the removal of
the filtering track from the last TREC conferences leaves a gap in the evaluation
of adaptive IF systems.
4.1
Evaluation Methodology
The evaluation methodology uses virtual users and a variation of the routing
subtask of the 10th TREC's (TREC-2001) filtering task
4
. TREC-2001 adopts
the Reuters Corpus Volume 1 (RCV1), an archive of 806,791 English language
news stories
5
, which have been manually categorised according to topic, region,
and industry sector. The TREC-2001 filtering track is based on 84 out of the
103 RCV1 topic categories. Furthermore, it divides RCV1 into 23,864 training
stories and a test set comprising the rest of the stories.
Since changes in a user's interests are reflected by variations in the distribution
of feedback documents about different topics, then we may simulate a virtual
user's interests in the following way. Given RCV1's classification, a virtual user's
current interests may be defined as a set of topics (e.g.
R
1
/R
2
/R
3) [23]. A
radical, long-term change of interest may then be simulated by removing, or
adding, a topic to this set. For example, if the user is no longer interested in
4
For more details see: http://trec.nist.gov/data/t10 filtering/T10filter guide.htm
5
http://about.reuters.com/researchandstandards/corpus/index.asp
