Figure 12.

Dependence of classification error (the ratio of erroneously classified to all examples) on the number of decision tree nodes. Red – not trimmed trees, green – trimmed decision trees and black – optimal trees. The selection of optimum trees consists in the minimisation of the terminal nodes number at parallel minimisation of the classification error. The optimum trees are those, which are marked black and their terminal nodes number ranges between 16 and 22. Optimum has been chosen in the place where the error is not reduced and the number of terminal nodes is the smallest. A shift in any direction affects the inflated number of terminal nodes, or substantially increases the error. The results marked in red show that non-trimmed decision trees over-fit to the data reaching a smaller error at the expense of the number of terminal nodes.