Adaptive Information Filtering - Text Mining: Classification, Clustering, and Applications - page 207

for the task of filtering (40) (11) (39) (42) (70). (40) suggested a list of
potential implicit feedbacks. (11) built a personal news agent that used time-
coded feedback from the user to learn a user profile. (39) investigated implicit
feedback for filtering newsgroup articles.
8.6.3 Exploration and Exploitation Trade Off
Most of the filtering systems deliver a document if and only if the expected
immediate utility of delivering it is greater than the expected utility of not
delivering it. However, delivering a document to the user has two effects: 1)
it satisfies the user's information need immediately, and 2) it helps the system
better satisfy the user in the future by learning from the relevance feedback
about this document provided by the user. An adaptive information filtering
approach is not optimal if it fails to recognize and model this second effect.
Some researchers have followed this direction. (15) considers exploration ben-
efit while filtering and carries out exploration and exploitation trade-off. (75)
studies the second aspect and models the long term benefit of delivering a
document as expected utility improvement as a result of improved model.
However, exploration and exploitation trade off is a problem far from being
solved.
8.6.4 Evaluation beyond Topical Relevance
Utility is an approximation of the user's criteria of a good document. Given
a utility measure, a system can achieve the objective of maximizing the user's
satisfaction through utility maximization using mathematical or statistical
techniques. A good utility measure is critical, because no system can do well
with an inappropriate objective. In the IR community, utility is usually de-
fined over relevance. Relevance was meant to represent a document's ability
to satisfy the needs of a user. However, this concept is very abstract and hard
to model, thus usually reduced to a narrow definition of “topical relevance”
or “related to the matter at hand (aboutness)” (45) (59). On the other hand,
“presenting the documents in order of estimated relevance” without consid-
ering the incremental value of a piece of information is not appropriate (58).
Researchers have studied criteria such as information-novelty for retrieval (17),
summarization (14), filtering (73), and topic detection and tracking (4). Prior
research on what is a user's perception/criteria has found that a wide range
of factors (such as personal knowledge, topicality, quality, novelty, recency,
authority, and author qualitatively) affect human judgments of relevance (8)
(36) (56) (60) (53). We also discussed how to estimate novelty in this chap-
ter, which is just an example of many of the important criteria for the user
besides relevance, such as readability (18) and authority (28). How to build
and evaluate a filtering system to optimize a more complex user criteria that
goes beyond “topical relevance” or “aboutness” is still a challenging research
problem for the adaptive filtering community.