Lessons Learned from Data Competitions: Data Leakage and Model Evaluation - Doing Data Science - page 314

Figure 13-3. This classic image from Hastie and Tibshirani's Elements
of Statistical Learning (Springer-Verlag) shows fitting linear regres‐
sion to a binary response, fitting 15-nearest neighbors, and fitting
1-nearest neighbors all on the same dataset
The picture on the left is underfit, in the middle it's good, and on the
right it's overfit.
The model you use matters when it concerns overfitting, as shown in
Figure 13-4 .
Figure 13-4. The model you use matters!
Looking at Figure 13-4 , unpruned decision trees are the overfitting-
est (we just made that word up). This is a well-known problem with
unpruned decision trees, which is why people use pruned decision
trees.