Beyond Classification Tasks - Data Mining with Decision Trees: Theory and Applications - page 85

Chapter 8

Beyond Classification Tasks

8.1 Introduction

Up to this point, we have mainly focused on using decision trees for
classification tasks (i.e. classification trees). However, decision trees can
be used for other data mining tasks. In this chapter, we review the
other common tasks that decision trees are used for, including: regression,
clustering and survival analysis.

8.2 Regression Trees

Regression models map the input space into a real-value domain. For
instance, a regressor can predict the demand for a certain product given its
characteristics. Formally, the goal is to examine y | X for a response y and

a set of predictors X .
Regression trees are decision trees that deal with a continuous target.
The basic idea is to combine decision trees and linear regression to forecast
numerical target attribute based on a set of input attributes. These methods
perform induction by means of an ecient recursive partitioning algorithm.
The choice of the best split at each node of the tree is usually guided by a
least squares error criterion.
Regression tree models are successfully used for day ahead forecasting
of power consumption using input features such as temperature, day of the
week and location.
CART and CHAID regression tree algorithms assign a constant value
to the leaves. The constant value is usually the mean value of the
corresponding leaf. RETIS and M5 algorithms are able to use linear
regression models at the leaves. Linear regression is a well-known statistics
method that was studied rigorously.

85