Utility-Based Information Distillation - Text Mining: Classification, Clustering, and Applications - page 220

or 'Already Seen.' Only the highlighted pieces are used by the system to
update its model (“profile”) of the current query. Depending on the type of
user feedback, the system takes one of the following actions:

• If the feedback type is 'Relevant,' use the highlighted piece of text as a

positive example in the adaptation of the query profile, and also add it
to the user's history.

• If the feedback type is 'Not-relevant,' use the highlighted piece of text

as a negative example in the adaptation of the query profile, and also
add it to the user's history.

• If the feedback type is 'Already Seen,' do not use the text for positive

or negative feedback; just add it to the user history.

As soon as the query profile is updated, the system re-issues a search and
returns another ranked list of passages where the previously seen passages are
either removed or ranked low, based on user preference. For example, if the
user highlights '...ocials have posted a $100,000 reward for their capture...'
as relevant answer to the query “What steps have been taken so far?”, then
the highlighted piece is used as an additional positive training example in
the adaptation of the query profile. This piece of feedback is also added to
the user history as a seen example, so that the system will be unlikely to
place another passage mentioning '$100,000 reward' in the future at the top
of the ranked list. However, an article mentioning '...ocials have doubled the
reward money to $200,000...' might be ranked high since it is both relevant
to the (updated) query profile and novel with respect to the (updated) user
history. The user may modify the original queries or add a new query during
the process; the query profiles will be changed accordingly. Clearly, novelty
detection is very important for the utility of such a system because of the
iterative search. Without novelty detection, the old relevant passages would
be shown to the user repeatedly in each ranked list.
Through the above example, we can see the main properties of our new
framework for utility-based information distillation over temporally ordered
documents. Our framework combines and extends the power of adaptive
filtering (AF), ad hoc retrieval (IR) and novelty detection (ND). Compared
to standard IR, our approach has the power of incrementally learning long-
term information needs and modeling a sequence of queries within a task.
Compared to conventional AF, it enables a more active role of the user in
refining his or her information needs and requesting new results by allowing
relevance and novelty feedback via highlighting of arbitrary spans of text in
passages returned by the system.
Compared to past work, this is the first evaluation of ND in a utility-
based framework, integrated with adaptive filtering for sequenced queries that
allows flexible user feedback over ranked passages. The combination of AF,
IR and ND with the new extensions raises an important research question