Evaluation of Classification Trees - Data Mining with Decision Trees: Theory and Applications - page 44

4.2.6.7 Average Hit-Rate

The average hit-rate is a weighted average of all hit-rate values. If the model
is optimal, then all the really positive instances are located in the head of
the ranked list, and the value of the average hit-rate is 1. The use of this
measure fits an organization that needs to minimize type II statistical error
(namely, to include a certain object in the quota although in fact this object
will be labeled as “negative”). Formally, the Average Hit-Rate for binary
classification problems is defined as:

n

t [ j ] · Hit - Rate ( j )

j =1

Average Hit - Rate = ,

(4.18)

n +

where t [ j ] is defined as in Equation (4) and is used as a weighting factor.
Note that the average hit-rate ignores all hit-rate values on unique positions
that are actually labeled as “negative” class (because t [ j ] =0 in these cases).

4.2.6.8 Average Qrecall

Average Qrecall is the average of all the Qrecalls which extends from the
position that is equal to the number of positive instances in the test set
to the bottom of the list. Average Qrecall fits an organization that needs
to minimize type I statistical error (namely, not including a certain object
in the quota although in fact this object will be labeled as “positive”).
Formally, average Qrecall is defined as:

n

Qrecall ( j )

j = n +

, (4.19)

n − ( n + − 1)

where n is the total number of instances and n + is defined in Equation
(4.13).
Table 4.2 illustrates the calculation of average Qrecall and average hit-
rate for a dataset of 10 instances. The table presents a list of instances
in descending order according to their predicted conditional probability to
be classified as “positive”. Because all probabilities are unique, the third
column ( t [ k ] ) indicates the actual class (“1” represent “positive” and “0”
represents “negative”). The average values are simple algebraic averages of
the highlighted cells.
Note that both average Qrecall and average hit-rate get the value 1 in
an optimum classification, where all the positive instances are located at