Database Reference
In-Depth Information
internal nodes. The left-hand side (LHS) internal node splits on a question based
on the
Income
variable to create leaf nodes at depth 2, whereas the right-hand side
(RHS) splits on a question on the
Age
variable.
The decision tree in
Figure 7.1
shows that females with income less than or equal
to $45,000 and males 40 years old or younger are classified as people who would
purchase the product. In traversing this tree, age does not matter for females, and
income does not matter for males.
Decision trees are widely used in practice. For example, to classify animals,
questions (like cold-blooded or warm-blooded, mammal or not mammal) are
answered to arrive at a certain classification. Another example is a checklist of
symptoms during a doctor's evaluation of a patient. The artificial intelligence
engine of a video game commonly uses decision trees to control the autonomous
actions of a character in response to various scenarios. Retailers can use decision
trees to segment customers or predict response rates to marketing and promotions.
Financial institutions can use decision trees to help decide if a loan application
should be approved or denied. In the case of loan approval, computers can use
the logical
if-then
statements to predict whether the customer will default on
the loan. For customers with a clear (strong) outcome, no human interaction is
required; for observations that may not generate a clear response, a human is
needed for the decision.
By limiting the number of splits, a short tree can be created. Short trees are
often used as
components
(also called
weak learners
or
base learners
) in
ensemble methods
. Ensemble methods use multiple predictive models to vote,
and decisions can be made based on the combination of the votes. Some popular
ensemble methods include random forest [4], bagging, and boosting [5]. Section
7.4 discusses these ensemble methods more.
The simplest short tree is called a
decision stump
, which is a decision tree
with the root immediately connected to the leaf nodes. A decision stump makes
a prediction based on the value of just a single input variable.
Figure 7.2
shows a
decision stump to classify two species of an iris flower based on the petal width.
The figure shows that, if the petal width is smaller than 1.75 centimeters, it's Iris
versicolor; otherwise, it's
Iris virginica
.
