Getting Started - Java Data Mining: Strategy, Standard, and Practice - page 123

produces rules; this was chosen by the DME, as the default algo-
rithm. Now that we have a model, we need to understand how well
it predicts whether a customer will respond to the campaign.

6.4.2 Test

For model testing, we use the CUSTOMER_TEST dataset. As the
code below indicates, we first create a physical dataset object from
the CUSTOMER_TEST data, and then create a classification test task
which specifies the test dataset, the previously built model, called
response_model , and the name of the test results. We accept the default
set of tests performed by the DME, which includes computing a
confusion matrix, lift, and ROC. After executing the task, we can
examine the lift to understand how well the model performs over a
random selection of customers.

// Create the physical representation of the input data for apply
PhysicalDataSetFactory pdsFactory (PhysicalDataSetFactory)
dmeConn.getFactory( ”javax.datamining.data.PhysicalDataSet” );
PhysicalDataSet testInputData pdsFactory.create( ”CUSTOMER_TEST”, true );
dmeConn.saveObject( ”cust_response_test”, testInputData, REPLACE );

// Create a task to run test operation
ClassificationTestTaskFactory cttFactory (ClassificationTestTaskFactory)
dmeConn.getFactory
( ”javax.datamining.supervised.classification.ClassificationTestTask” );
ClassificationTestTask testTask cttFactory.create(
”cust_response_test”, ”resp_model_1”, ”resp_model_1_test_result” );

// Enable computation of confusion matrix as the result of test
testTask.computeMetric(ClassificationTestMetricOption.confusionMatrix);
dmeConn.saveObject( ”response_test_task_1”, testTask, false );

// Execute the task asynchronously, but wait until done
ExecutionHandle execHandle dmeConn.execute( ”response_test_task_1” );
execHandle.waitForCompletion( WAIT_FOR_COMPLETION );

// Retrieve the test metrics
ClassificationTestMetrics testMetrics (ClassificationTestMetrics)
dmeConn.retrieveObject( ”resp_model_1_test_result”,
NamedObject.testMetrics );
Double accuracy testMetrics.getAccuracy();
Lift lift testMetrics.getLift();
Double cumulativeLift lift.getCumulativeLift (4);

From this default model, the cumulative lift for the test dataset at
40 percent is .6, meaning that the model was able to predict 60 percent