Cost-sensitive Active and Proactive Learning of Decision Trees - Data Mining with Decision Trees: Theory and Applications - page 191

for rejection is particularly undesirable. Moreover, predictive accuracy alone
does not provide enough flexibility when selecting a target for a marketing
offer or when choosing how an offer should be promoted. For example, the
marketing personnel may want to approach 30% of the available potential
customers, but the model predicts only 6.
Active learning merely aims to minimize the cost of acquisition, and
does not consider the exploration/exploitation tradeoff. Active learning
techniques do not aim to improve online exploitation. Nevertheless, occa-
sional income is a byproduct of the acquisition process. We propose that the
calculation of the acquisition cost performed in active learning algorithms
should take this into consideration.
Several active learning frameworks are presented in the literature. In
pool-based active learning the learner has access to a pool of unlabeled data
and can request the true class label for a certain number of instances in the
pool. Other approaches focus on the expected improvement of class entropy,
or minimizing both labeling and misclassification costs. It is possible to
examine a variation in which instead of having the correct label for each
training example, there is one possible label (not necessarily the correct one)
and the utility associated with that label. Most active learning methods
aim to reduce the generalization accuracy of the model learned from the
labeled data. They assume uniform error costs and do not consider benefits
that may accrue from correct classifications. They also do not consider the
benefits that may be accrued from label acquisition.
Rather than trying to reduce the error or the costs, Saar-Tsechansky
and Provost (2007) introduced the GOAL (Goal-Oriented Active Learning)
method that focuses on acquisitions that are more likely to affect decision
making. GOAL acquires instances which are related to decisions for which
a relatively small change in the estimation can change the preferred order
of choice. In each iteration, GOAL selects a batch of instances based on
their effectiveness score. The score is inversely proportional to the minimum
absolute change in the probability estimation that would result in a decision
different from the decision implied by the current estimation. Instead of
selecting the instances with the highest scores, GOAL uses a sampling
distribution in which the selection probability of a certain instance is
proportional to its score.
To better understand the idea of active learning we consider a concrete
scenario. When marketing a service or a product, firms increasingly
use predictive models to estimate the customers' interest in their offer.
A predictive model estimates the response probability of the potential