Visual explainer

Decision Trees

Carve the data space into rectangles with a series of yes-or-no questions, predicting a single value for everyone in the same box.

When data is arranged in patches or clusters, a single straight line can't separate it.

Linear models fail when the data forms patches, islands, or complex shapes. You need a model that can draw corners and surround groups of points.

One Cut at a Time

Instead of a diagonal line, we make a single straight cut along one axis.

A decision tree doesn't use math formulas to bend a line. It just asks a yes-or-no question about one feature at a time, making a straight, axis-aligned cut through the data.

Splitting the Splits

The model repeats the process on each side, building a tree of questions.

Once the first cut is made, the tree looks at the left side and asks the best question for just those points. Then it looks at the right side and does the same, splitting the data again and again.

Carving the Space

These splits carve the space into rectangles, and each rectangle gets its own prediction.

If you look at the cuts from above, they carve the entire feature space into a grid of rectangles. Every new data point falls into exactly one box, and receives the prediction assigned to that box.

The Staircase Problem

Because trees can only cut vertically and horizontally, they must draw a jagged staircase to approximate a diagonal line.

Trees are fast and handle messy data well, but they have a blind spot. Because every cut must be strictly horizontal or vertical, they cannot draw a simple diagonal line. To separate data that runs diagonally, a tree is forced to build a deep, fragile staircase.

The Quick Version

Trees don't use formulas, they ask yes-or-no questions.
Every question makes one straight, axis-aligned cut.
The splits nest inside each other to form a tree.
The result is a space carved into pure rectangles.
They are forced to use jagged staircases for diagonal data.