Cost-sensitive Active and Proactive Learning of Decision Trees - Data Mining with Decision Trees: Theory and Applications - page 186

than overestimate, and that it is less costly to be slightly wrong than very
wrong. Cost matrix is different for cost vector, which has been used in some
previous learning research [ Provost (1994) ] . With a cost vector, the cost of
misclassifying an example depends on either the actual cost or the predicted
cost but not both. Any cost matrix C can be transformed to an equivalent
matrix C with zeroes along the diagonal [ Margineantu (2001) ] .
To reduce the cost of misclassification errors, some have incorporated
an average misclassification cost metric in the learning algorithm [ Pazzani
et al . (1994) ] :

i

N

C ( actualClass ( i ) ,predictedClass ( i ))

averageCost = (12.3)

N

Several decision trees inducers, such as C4.5 and CART can be provided
with a cost matrix which consists of numeric penalties for classifying an item
into one class when it really belongs in another.
Few algorithms are based on a hybrid of accuracy and classification
error cost [ Pazzani et al . (1994) ] replacing the information gain measure-
ment with a combination of accuracy and cost. For example, Information
Cost Function (ICF) selects attributes based on both their information gain
and their cost [ Turney (1995) ] . ICF for the i-th attribute, ICF i , is defined
as follows:

2 ∆ I i −

1
( C i +1) w

ICF i = (12.4)

where 0 1, ∆ I i is the information gain associated with the i-th

attribute at a given stage in the construction of the decision tree, and
C i is the cost of measuring the i-th attribute. The parameter w adjusts
the strength of the bias towards lower cost attributes. When w =0,cost
is ignored and selection by ICF i is equivalent to selection by ∆ I i (i.e.,
selection based on the information gain measure). When w =1, ICF i is
strongly biased by cost.
The altered prior method [ Biermann et al . (1982) ] ,whichworkswith
any number of classes, operates by replacing the term for the prior
probability, π ( j ), that an example belongs to class j with an altered
probability π ( j ):

≤ w ≤

C ( j ) π ( j )

π ( j )= (12.5)

C ( i ) π ( i )

i

where C ( j )=
i

cost ( j, i )