True Positive Rate
False Positive Rate
0 0.2 0.4 0.6 0.8 1
The ROC Curve of data points of Table 12-1.
these numbers is that they can be used to pick the probability thresh-
old to optimize the revenue if we know the cost associated with each
of the 3,715 negative responses and the profit associated with the
7,882 positive responses.
Given these numbers, we can select—for each model, based on the
ROC curve obtained on the test dataset—the probability threshold
that will maximize revenue. The following code shows how to per-
form that computation by creating a TestMetrics task for each model
and asking for the ROC curve, which is specified at line 20. We scan
through the different threshold candidates of the ROC curve. Each
threshold candidate provides access to the elements shown in Table
12-1. In particular, we can get back the number of true positives and
false positives, and compute the expected profit if we set the thresh-
old at this level. The expected return will be the difference between
the number of true positives (the customers that are contacted and
responded positively) times the individual profit, and the number of
false positives (the customers that are contacted but did not respond
positively) times the individual cost.
(number of true positives
(number of false positives