Visual explainer

Decision Trees

Follow ten illustrative samples through a Gini-scored split, recursive branches and an unseen-point prediction, then inspect depth and diagonal boundaries.

Ten illustrative samples mix two classes across two features.

A classification tree predicts a class by following feature comparisons to a leaf. These ten illustrative samples have two numeric features, x and y, and classes A and B. Their coordinates stay fixed through the split diagrams; the shapes identify classes independently of colour.

One Question, One Cut

The threshold x ≤ 30 sends five samples to each child.

The x threshold partitions the samples into a left child containing 4 A and 1 B, and a right child containing 1 A and 4 B. Equality goes left. This axis-aligned split tests one feature; it doesn't yet separate every class.

Impurity Falls 0.18

The split reduces weighted Gini impurity from 0.50 to 0.32.

Gini impurity is one minus the sum of squared class proportions. The root gives 1 − 0.5² − 0.5² = 0.50. Each child gives 1 − 0.8² − 0.2² = 0.32. Weighting children by their sample counts gives 0.32, a reduction of 0.18. A greedy tree picks the candidate with the lowest weighted child impurity.

Recurse Inside Each Child

Three split nodes create four pure leaves at depth two.

Apply y ≤ 50 inside each child. Four leaves contain 4 A, 1 B, 4 B and 1 A. There are three split nodes, but a sample encounters only two comparisons. The chosen threshold lies between observed y values 40 and 90; it isn't a claimed library fit.

Trace a New Point

An unseen point follows two comparisons to one predicted class.

For the illustrative point (46,70), x > 30 chooses the right subtree and y > 50 reaches its A leaf. A is the prediction; the point's true class is unknown. A pure training leaf doesn't establish correctness on unseen samples.

Stop or Split Again

Stopping at depth one leaves two training mistakes in this example.

Stopping after the x split predicts each child's majority class and misses one sample per child: 2/10. Depth two fits all ten training labels. Choose depth, leaf-size limits or pruning using held-out evidence; lower training error alone doesn't settle the choice.

Where It Breaks

Axis-aligned leaves approximate a diagonal boundary with steps.

This separate idealized example uses the diagonal boundary y = x. Rectangular leaves form a staircase approximation; more splits can refine it, adding complexity. Useful depth depends on the geometry and tolerated error. Recursive partitioning and its stopping rules define the model; validation determines whether added detail helps.

The Quick Version

Classification trees follow feature comparisons to a predicted class.
The x threshold splits this fixture into two five-sample children.
Weighted Gini falls from 0.50 to 0.32.
Three split nodes form four leaves at depth two.
An unseen point's predicted class can differ from its true class.
Training fit alone doesn't select depth or pruning.
Axis-aligned leaves approximate diagonals with rectangular steps.