Discretisation and Binning
Binning throws away resolution on purpose. Sometimes that buys you a step a linear model could never fit; hand it to a tree and you have only lost detail.
Why Does This Exist?
Age 34 becomes "30 to 39". You just deleted most of what that column knew, on purpose, and nothing raised an error.
Start from the position that this is usually the wrong move. A gradient-boosted tree already finds thresholds in a continuous column, by optimising your actual loss, and it can pick different ones in different branches. Four hand-chosen edges will not beat that. Binning before a boosted model is a net loss, and a common habit.
So the page has to earn it. There are cases where losing resolution buys something a continuous column cannot, and there are choices of edges where the same four bins behave completely differently. Binning also moves a column across the numerical and categorical split by your decision rather than by its nature. This page covers when it pays, how to choose the edges, and the two traps: the boundary artefact and leaking through supervised edges.
Think of It Like This
Shoe sizes and tax brackets
Feet are continuous. Shoes come in sizes. Your foot is 27.4cm and you get a 27, and every shop does it anyway, because a shelf of forty sizes is one you can run and infinite sizes is not.
Now tax. Earn one pound more, cross a threshold, and your marginal rate jumps. There is no smooth curve underneath: the step is the rule. A straight line through tax owed against income is wrong on both sides of every threshold whatever coefficient you give it. Bin at the bracket edges and it gets it right.
Two reasons to bin, and only the second is about accuracy. Neither applies to a model already picking its own thresholds.
Where it stops: a shoe size still orders feet, while a bin index fed to a model as one-hot forgets the order entirely. Sizes 27 and 28 stay neighbours on the shelf. Bins 1 and 2 can end up as unrelated as bins 1 and 4.
How It Actually Works
Discretisation maps a continuous column onto a small set of ordered labels.
When binning actually pays
A step a straight line cannot make. Claim risk jumps at 65; shipping cost changes at 30kg. A linear model averages across the step and gets both sides wrong. A relationship that turns around: high for the very young, low in the middle, high again. No single coefficient fits that, but one parameter per band does. A rule already written in bands: "income band 3" is auditable, and a five-decimal income is not. Extremes you would rather not delete: a top bin of "80 and over" keeps the rows and the fact they are extreme, without outlier removal. Something you can watch: four band proportions drift visibly, while a continuous distribution needs a distance measure and a threshold.
Choosing the edges
Equal-width cuts the range into intervals of the same size. It is what bins=4 gives you, and on skewed data most rows land in one or two bins.
Equal-frequency, or quantile binning, puts the same row count in every bin and lets the widths fall where they must. Usually what you want, because every bin then means something.
K-means binning takes cluster boundaries from 1-D k-means. Right when the column is genuinely multi-modal.
Supervised binning picks cut points from the target: fit a shallow tree on that one column and lift its thresholds. Credit scoring formalises it as weight-of-evidence binning, taking the log of the ratio of positives to negatives in each candidate bin and merging until every bin is monotone. Summing the weighted contributions gives an information value, one number for how well the column separates the classes.
That reads the target, so it leaks like target encoding. Fit the edges inside each fold, never on the whole frame, or you have built data leakage into an innocent-looking feature.
Encoding the bins, and the boundary problem
Bin index 0, 1, 2, 3 is ordinal and keeps the order, so a monotone effect needs one coefficient. One-hot instead and each band gets a free parameter, which is what makes the non-monotone case work. You pay in rows per coefficient.
An edge at 40 makes 39.9 and 40.1 different categories, and one-hot them and they are as unrelated as 39.9 and 91. It is worst where the data is densest, and quantile edges land there by construction. If the step needs to be soft, you wanted a transform instead, such as a spline.
Worked example
Twenty ages, mostly twenties and thirties with a thin older tail: 22, 24, 25, 27, 28, 29, 31, 32, 33, 34, 35, 36, 38, 41, 44, 52, 58, 67, 74, 88.
Four equal-width bins. The range runs 22 to 88, so each is 16.5 years wide, with interior edges at 38.5, 55 and 71.5. Counts: 13, 3, 2, 2.
Four equal-frequency bins. Quartile edges land at 28.75, 34.5 and 46. Counts: 5, 5, 5, 5, with widths of 6.75, 5.75, 11.5 and 42 years. The bottom bin covers under seven years because that is where everyone is; the top covers forty-two.
Same column, same number of bins, and one version has three bins that cannot estimate anything.
Code
import numpy as np
ages: np.ndarray = np.array([22, 24, 25, 27, 28, 29, 31, 32, 33, 34, 35, 36, 38, 41, 44, 52, 58, 67, 74, 88], dtype=float)
wide_counts, wide_edges = np.histogram(ages, bins=4) # equal widthprint(wide_edges.tolist()) # -> [22.0, 38.5, 55.0, 71.5, 88.0]print(wide_counts.tolist()) # -> [13, 3, 2, 2]
q_edges: np.ndarray = np.percentile(ages, [0, 25, 50, 75, 100])freq_counts, _ = np.histogram(ages, bins=q_edges) # equal frequencyprint(q_edges.tolist()) # -> [22.0, 28.75, 34.5, 46.0, 88.0]print(freq_counts.tolist()) # -> [5, 5, 5, 5]
def assign(value: float, edges: np.ndarray) -> int: return int(np.clip(np.digitize(value, edges[1:-1]), 0, len(edges) - 2))
print(assign(34.4, q_edges), assign(34.6, q_edges)) # -> 1 2The last line is the boundary artefact: two ages a fifth of a year apart, in different bands.
Watch Out For
Binning a column and then handing it to a tree
KBinsDiscretizer into a gradient-boosting classifier, or a pd.cut on age followed by XGBoost. Written constantly, and it can only lose.
The tree was already going to search that column. It scores every candidate split point against your loss and picks per node, so a deep tree can use 40 in one branch and 62 in another. You replaced that with four fixed edges chosen without ever looking at the target.
Symptom: the binned model underperforms the raw-column model by a little on every split, forever. Fix: do not bin for trees. If the reason is "it reduces noise", reach for a transform instead.
Edges fitted on training data meeting a value from outside the range
Your bin edges came from a training set where age topped out at 88. A 94-year-old arrives, pd.cut returns NaN, and the row crashes or gets imputed. KBinsDiscretizer clips into the outer bin instead, worth knowing rather than discovering. Set explicit outer edges at negative and positive infinity.
The other half is supervised edges. Weight-of-evidence bins and tree-derived thresholds are chosen using the target, so they are fitted state exactly like a target encoder, and computing them once makes every fold optimistic in the same direction.
Symptom: production NaNs on extreme values, or validation that flatters supervised bins. Fix: pin the outer edges open and refit supervised edges per fold.
The Quick Version
- Binning deletes resolution deliberately. Assume it is wrong until you can name why it is not.
- A tree finds its own thresholds, per node, against your loss.
- It pays for a step a linear model cannot fit, a non-monotone relationship, a rule written in bands, extremes you want kept, and a feature you monitor.
- Equal-width bins have equal widths and unequal counts, so skewed data gets empty top bins.
- Equal-frequency bins hold the same count and get wide in the tail. Usually the right default.
- Supervised binning picks edges from the target: tree thresholds, or weight of evidence.
- Ordinal indices keep the order. One-hot gives each band its own effect.
- An edge at 40 makes 39.9 and 40.1 unrelated, worst where the data is densest.
- Supervised edges leak unless refitted per fold, and an out-of-range value becomes a NaN.