Numerical and Categorical Data

Why Does This Exist?

A churn model ships with its largest coefficient sitting on country_id. Nobody blinks in the review, because it is a number and numbers are what models eat.

The column is an arbitrary integer from a lookup table someone populated alphabetically in 2019. The model has concluded that Belgium, which happens to be 7, contributes more than twice what Austria, which happens to be 3, does. Every distance, split threshold and gradient on that column is arithmetic on a label.

That is the whole subject. Measurement scale decides which arithmetic is meaningful, and the dtype says nothing about it. int64 is a storage decision. Whether adding two of those values means anything is a fact about the world.

Nearly every encoding bug is this mistake. Either a label got treated as a magnitude, or an ordering got thrown away because the column looked like a label. The fix for the first case is categorical encoding. The fix that turns a skewed count into usable groups is discretization and binning.

The one idea you need first. Columns split into labels you compare for equality and magnitudes you can order, subtract and average. Nominal means equality only, ordinal adds order, numeric adds gaps and ratios. The rest of the page is deciding which of the three you hold before any encoder or scaler touches it.

Think of It Like This

How It Actually Works

Two questions. Can the values be counted or must they be measured? And what does arithmetic on them mean?

The scales, and what each one permits

Discrete or continuous. Discrete values are countable: bedrooms, clicks, defects. Continuous ones are measured: mass, duration, temperature.

Nominal. Labels with no order. Country, browser, payment method. Equality is the only meaningful operation: a mean is nonsense, a greater-than is nonsense, a distance is nonsense.

Ordinal. Ordered labels with uneven gaps. Survey responses, sizes, severity, star ratings. Comparison and median are meaningful, a mean is a stretch you should be able to defend.

Binary. One bit. With two levels the only difference is 1, so a label and a number are the same object.

Counts. Discrete, floored at zero, usually skewed right, because you can have twenty children but never negative three. The mean drifts above the median, so a mean of 2.3 children is a real number describing nobody. A symmetric error model on that column will also predict negative counts, which is why counts want a log transform or binning.

Identifiers. user_id, sku, postcode. Strings that repeat, so an encoder takes them, but the cardinality makes them behave like free text: hundreds of thousands of levels, a handful of rows each, and a vocabulary that is your training set. Hash, aggregate, or drop.

Cyclical values wrap, and integers do not

Hour of day, month, compass bearing, phase. The values run in a circle, so 23:00 and 00:00 are one hour apart.

Store the hour as an integer and the model sees $|23 - 0| = 23$, the largest gap in the column, for two instants sixty minutes apart. Every late-night pattern gets cut in half at the wrap. The fix puts each value on a circle, two columns, $\sin(2\pi h/24)$ and $\cos(2\pi h/24)$, so hour 23 and hour 0 land beside each other by construction.

Worked example

Take size with values small, medium, large. Ordinal encoding maps them to 0, 1, 2 in one column, so a single threshold expresses every at-least-this-big question: size >= 1 is medium or larger, size >= 2 is large.

One-hot gives three columns and still isolates large with one threshold on size_large. What it cannot do is medium or larger without combining two, and it hands a linear model three unrelated coefficients where the ordering was the signal.

The cyclical case runs the other way. As integers, hour 23 to hour 0 is a distance of 23 and hour 0 to hour 1 is 1. On the circle both are chord lengths of $2\sin(\pi/24) \approx 0.261$, which is correct: both pairs are an hour apart.

Code

Both encodings, and the arithmetic each one permits.

1import numpy as np2
3order: dict[str, int] = {"small": 0, "medium": 1, "large": 2}4sizes: list[str] = ["small", "medium", "large", "medium"]5scale = np.array([order[s] for s in sizes], dtype=float)6
7
8def clock(hours: np.ndarray) -> np.ndarray:9    """Hours onto a circle, so 23:00 sits beside 00:00 instead of 23 away."""10    angle = 2.0 * np.pi * hours / 24.011    return np.stack([np.sin(angle), np.cos(angle)], axis=1)12
13ring: np.ndarray = clock(np.array([23, 0, 1]))14print(scale.tolist())  # -> [0.0, 1.0, 2.0, 1.0]15print((scale >= 1).tolist())  # -> [False, True, True, True]16print([abs(23 - 0), abs(0 - 1)])  # -> [23, 1]17print([round(float(v), 3) for v in [np.linalg.norm(ring[0] - ring[1])]])  # -> [0.261]

One threshold selects medium and up. The integer distance is the bug, the chord length the fix.

Watch Out For

The Quick Version

• Measurement scale, not dtype, decides which arithmetic on a column is meaningful.
• Nominal permits equality. Ordinal permits order and a median. Interval and ratio permit sums, means and ratios.
• Discrete values are counted, continuous ones measured, and the split changes how you plot them.
• Counts sit at zero and skew right, so the mean drifts above the median and describes nobody.
• Cyclical values wrap. As integers, 23:00 and 00:00 sit furthest apart in the column. Use sine and cosine.
• An ordinal column one-hot encoded loses the ordering, which was the signal you had.
• A nominal column left as int64 reads as a magnitude to every coefficient and distance downstream.
• High-cardinality identifiers look categorical and behave like free text, vocabulary and all.

What to Read Next

• Categorical Encoding
• Discretisation and Binning
• Feature Scaling