Data Concepts

Categorical Encoding

Count the distinct values first. Two, a dozen, or fifty thousand — that number decides how the column becomes numbers, and taste has nothing to do with it.

One categorical column can become one ordinal integer, a handful of one-hot columns, or a fixed-width hash, and the number of distinct values is what picks between them

Why Does This Exist?

ValueError: could not convert string to float: 'united-kingdom'

Models consume numbers. Half of every real dataset is words: country, plan tier, device, merchant category, user ID. Something has to convert one into the other, and how it converts decides what the model can infer.

The wrong choice fails quietly. Map blue=1, green=2, teal=3 and the code runs. You've also told a linear model that teal is three times blue and green sits between them, which is nonsense it will faithfully fit.

The decision is nearly mechanical, though. Count the distinct values, and the count picks the method. Cardinality is the name for that count, and the first thing to check on any categorical column.

Think of It Like This

A dial, a row of switches, or neither

You have to describe a colour to a machine that only understands switches.

A dial numbered 1 to 4 works when the thing has an order. Small, medium, large: turn it to 3 and "more than 2" is meaningful.

Use the same dial for four unrelated colours and you've lied to it. The dial says 4 is twice 2, so the fourth colour is now twice the second.

A row of four switches, one per colour, exactly one flipped on. No invented order. Four colours, four switches, fine. Fifty thousand product IDs and you have a wall nobody can build.

Switches stop being the answer somewhere around fifteen.

How It Actually Works

Two values. One column, 0 and 1. Nothing to decide.

Three to about fifteen, unordered. One-hot: one binary column per value, exactly one hot per row. No invented ordering, and every model reads it correctly.

Genuinely ordered, any count. Ordinal encoding — integers that respect the order, small=0, medium=1, large=2. Do it on purpose: the order is signal and one-hot discards it. Sizes are ordered; countries never are.

Hundreds. You need a width that doesn't grow with the vocabulary. Frequency encoding replaces each category with how often it appears. Target encoding replaces it with the mean target for that category — powerful, and the most reliable way to leak labels into features, so compute it out-of-fold. The hashing trick maps categories into $k$ buckets you choose, so the width is fixed whatever arrives; the cost is collisions the model can't split.

Thousands, with a model that can learn them. A learned embedding: a short dense vector per category, trained alongside the network.

Why one-hot breaks at high cardinality

Two independent reasons.

Width. 50,000 categories, 50,000 columns. Dense float64 over a million rows is 400GB, which is why OneHotEncoder returns a sparse matrix and why .toarray() before a sparse-capable model is how a pipeline runs out of memory. And a linear model now holds more parameters than rows, so it fits the training set exactly and learns nothing.

Statistics. Each column is almost entirely zeros. A category appearing in 12 rows out of a million gives its coefficient 12 rows of evidence, so the fitted value is mostly noise — and there are 50,000 of them. That's the curse of dimensionality through your encoder.

The mechanics people get wrong

The dummy-variable trap. With $k$ one-hot columns and an intercept the columns sum to 1 in every row, which is what the intercept already is, so the matrix is rank-deficient and the coefficients aren't unique. drop_first=True fixes it, and it matters for unpenalised linear models, not trees.

Trees split badly on wide one-hot blocks. Each binary column holds a thin slice of the signal, so a greedy split on any one looks weak beside a continuous feature and the tree ignores the block. Use LightGBM or CatBoost's native handling instead.

Worked example

Two columns: colour with four values, size with three that are genuinely ordered. Take colour="green", size="large".

One-hot the colour against the sorted vocabulary [amber, blue, green, teal] and you get [0, 0, 1, 0]: four columns, one hot, no claim about which is larger.

Ordinal-encode the size against [small, medium, large] and "large" becomes 2. That 2 carries weight: size > 1 selects exactly the large rows, which a tree finds in one question.

Integer-encode the colour the same way and "green" also becomes 2 — asserting green outranks blue, and teal is the largest colour there is.

Show Me the Code

1import numpy as np2
3colours: list[str] = ["amber", "blue", "green", "teal"]   # nominal, no order to preserve4sizes: list[str] = ["small", "medium", "large"]           # ordered, and that order is signal5
6def one_hot(value: str, vocab: list[str]) -> np.ndarray:7    out = np.zeros(len(vocab), dtype=np.int8)8    if value in vocab:                    # an unseen value leaves the row all zeros9        out[vocab.index(value)] = 110    return out11
12def ordinal(value: str, order: list[str]) -> int:13    return order.index(value)             # the list order IS the encoding, so order it on purpose14
15print(one_hot("green", colours))          # -> [0 0 1 0]16print(ordinal("large", sizes))            # -> 217print(one_hot("cyan", colours))           # -> [0 0 0 0]  unseen at serving, silent, scored anyway18print(colours.index("green"))             # -> 2  as one integer, this claims green outranks blue

The third line is the production failure, the fourth the modelling one. Both return without complaint.

Watch Out For

An unseen category arriving at serving time

Your encoder's vocabulary was fixed the moment you called fit. A new merchant signs up, a device string changes, a country gets renamed, and the encoder meets a value it has no column for.

Two behaviours, neither good. It raises and the request fails, or handle_unknown="ignore" emits an all-zero row the model scores into a confident number, because all-zeros is a real point in the input space. Nothing logs.

Collapse rare categories into an __other__ bucket at training time with a frequency floor, so the bucket is fitted from real rows. Track the unseen-category rate, because a rising one is data drift.

Label-encoding a nominal column into one integer

LabelEncoder is built for targets. It lands on features constantly, because it's one line and the error message goes away.

Map 40 countries to 0 through 39 and hand it to a linear model: you've claimed country 39 is thirty-nine times country 1. Give it to KNN and country 5 sits 2 away from country 7 and 30 from country 35 — an alphabetical accident deciding which rows are neighbours.

Trees survive it better than people expect: enough splits carve out any subset. They pay in depth.

The Quick Version

Cardinality picks the method. Count the distinct values first.
Two values: one binary column. Three to fifteen unordered: one-hot. Ordered: integers respecting the order.
Hundreds: frequency encoding, out-of-fold target encoding, or hashing to bound the width. Thousands: an embedding.
One-hot on 50,000 values fails twice: on width, and on rows-per-coefficient.
drop_first matters for unpenalised linear models, not trees. One-hot output is sparse, so keep it sparse.
An unseen category at serving crashes or produces a silent all-zero row. Build an __other__ bucket from a frequency floor.
A nominal column squeezed into one integer asserts an order that isn't there, and a linear model or distance metric believes it.