Data Concepts

Numerical and Categorical Data

What kind of arithmetic a column actually allows. Get that wrong and a model will happily average country codes, or decide midnight is nowhere near 11pm.

One column, one question about its measurement scale, and three answers that each permit different arithmetic: equality only, order and a median, or sums and ratios, with the nominal branch being the one an integer column silently escapes

Why Does This Exist?

A churn model's largest coefficient sits on country_id. Nobody blinks in the review, because it's a number and numbers are what models eat.

The column is an arbitrary integer from a lookup table someone populated alphabetically in 2019. The model has concluded that Belgium, which happens to be 7, contributes more than twice what Austria, which happens to be 3, does. Every distance, split threshold and gradient on that column is arithmetic on a label.

That's the whole subject. Measurement scale decides which arithmetic is meaningful, and the dtype says nothing about it. int64 is a storage decision. Whether adding two of those values means anything is a fact about the world.

Nearly every encoding bug is this mistake. Either a label got treated as a magnitude, or an ordering got thrown away because the column looked like a label.

Think of It Like This

Squad numbers, finishing places, finishing times

Three columns of numbers off one afternoon at a running track. All integers. All meaningless in different amounts.

The squad numbers. Number 9 is not three times number 3, and the average shirt in a photograph isn't a fact about anything. The only question they answer is "same runner or different runner?"

The finishing places. Order means something now: 1st beat 2nd, who beat 3rd. The gaps don't. The winner might have been ten seconds clear while 2nd and 3rd crossed together, so 2 minus 1 and 3 minus 2 are different distances. Rank them, take a median, not a mean.

The finishing times. Order, gaps and ratios all mean something. 24 minutes really is twice 12. Add, average, subtract, and every answer is about the world.

Same digits, three sets of permitted operations. Nothing in the file tells you which one you're holding.

How It Actually Works

Two questions. Can the values be counted or must they be measured? And what does arithmetic on them mean?

The scales, and what each one permits

Discrete or continuous. Discrete values are countable: bedrooms, clicks, defects. Continuous ones are measured: mass, duration, temperature.

Nominal. Labels with no order. Country, browser, payment method. Equality is the only meaningful operation: a mean is nonsense, a "greater than" is nonsense, a distance is nonsense.

Ordinal. Ordered labels with uneven gaps. Survey responses, sizes, severity, star ratings. Comparison and median are meaningful; a mean is a stretch you should be able to defend.

Binary. One bit. With two levels the only difference is 1, so a label and a number are the same object.

Counts. Discrete, floored at zero, usually skewed right, because you can have twenty children but never negative three. The mean drifts above the median, so a mean of 2.3 children is a real number describing nobody. A symmetric error model on that column will also predict negative counts, which is why counts want a log transform or binning.

Identifiers. user_id, sku, postcode. Strings that repeat, so an encoder takes them, but the cardinality makes them behave like free text: hundreds of thousands of levels, a handful of rows each, and a vocabulary that is your training set. Hash, aggregate, or drop.

Cyclical values wrap, and integers don't

Hour of day, month, compass bearing, phase. The values run in a circle, so 23:00 and 00:00 are one hour apart.

Store the hour as an integer and the model sees $|23 - 0| = 23$ , the largest gap in the column, for two instants sixty minutes apart. Every "late night" pattern gets cut in half at the wrap. The fix puts each value on a circle, two columns, $\sin(2\pi h/24)$ and $\cos(2\pi h/24)$ , so hour 23 and hour 0 land beside each other by construction.

Worked example

Take size with values small, medium, large. Ordinal encoding maps them to 0, 1, 2 in one column, so a single threshold expresses every "at least this big" question: size >= 1 is medium or larger, size >= 2 is large.

One-hot gives three columns and still isolates large with one threshold on size_large. What it can't do is "medium or larger" without combining two, and it hands a linear model three unrelated coefficients where the ordering was the signal.

The cyclical case runs the other way. As integers, hour 23 to hour 0 is a distance of 23 and hour 0 to hour 1 is 1. On the circle both are chord lengths of $2\sin(\pi/24) = 0.261$ , which is correct: both pairs are an hour apart.

Show Me the Code

Both encodings, and the arithmetic each one permits.

1import numpy as np2
3order: dict[str, int] = {"small": 0, "medium": 1, "large": 2}4sizes: list[str] = ["small", "medium", "large", "medium"]5scale = np.array([order[s] for s in sizes], dtype=float)6
7
8def clock(hours: np.ndarray) -> np.ndarray:9    """Hours onto a circle, so 23:00 sits beside 00:00 instead of 23 away."""10    angle = 2.0 * np.pi * hours / 24.011    return np.stack([np.sin(angle), np.cos(angle)], axis=1)12
13
14ring: np.ndarray = clock(np.array([23, 0, 1]))15circ = [round(float(np.linalg.norm(ring[i] - ring[j])), 3) for i, j in [(0, 1), (1, 2)]]16print(scale.tolist())             # -> [0.0, 1.0, 2.0, 1.0]17print((scale >= 1).tolist())      # -> [False, True, True, True]18print([abs(23 - 0), abs(0 - 1)])  # -> [23, 1]19print(circ)                       # -> [0.261, 0.261]

One threshold selects medium and up. The integer distance is the bug, the chord length the fix.

Watch Out For

A nominal column stored as an integer

country_id, store_id, category_code, cluster_label. Pandas reads the column, sees digits, and gives you int64. Nothing warns you, because nothing knows.

Downstream, everything treats it as a magnitude. A linear model fits one coefficient, so it's asserting churn rises steadily as country_id rises, which is a claim about alphabetical order. A distance metric puts country 3 nearer country 4 than country 40. A tree survives best, since it can carve the range into pieces, but it burns depth doing with many splits what one categorical split would do.

Two habits catch it. On load, list every integer column and ask whether the difference between two values means anything. Then check cardinality against the row count: 200 distinct values over 50,000 rows with no arithmetic behind them is a label, whatever the dtype says.

Cast it, then encode it. If the column is genuinely ordinal, keep the integers and note why in a comment, so the next person doesn't "fix" it.

Cyclical features encoded as a straight line

df["hour"] = df.timestamp.dt.hour is one of the most-typed lines in applied machine learning, and it puts midnight as far as possible from 23:00.

The damage is specific. A model looking for a late-night effect has to learn two disconnected regions, at the very top and the very bottom of the range, so it needs roughly twice the data and gets a weaker signal from each half. A tree can approximate the wrap with hour >= 22 and hour <= 1, and often finds it, which is why the bug survives in production. A linear model or a distance metric can't recover it at all: the coefficient is forced monotone in hour, and the wrap is not.

Same story for month (December next to January), day_of_week, bearings, and any phase angle.

Two columns per cyclical field, $\sin(2\pi v/p)$ and $\cos(2\pi v/p)$ for period $p$ . Two lines of code, one extra column, and the wrap becomes true by construction instead of something a model has to notice.

The Quick Version

Measurement scale, not dtype, decides which arithmetic on a column is meaningful.
Nominal permits equality. Ordinal permits order and a median. Interval and ratio permit sums, means and ratios.
Discrete values are counted, continuous ones measured, and the split changes how you plot them.
Counts sit at zero and skew right, so the mean drifts above the median and describes nobody.
Cyclical values wrap. As integers, 23:00 and 00:00 sit furthest apart in the column. Use sine and cosine.
An ordinal column one-hot encoded loses the ordering, which was the signal you had.
A nominal column left as int64 reads as a magnitude to every coefficient and distance downstream.
High-cardinality identifiers look categorical and behave like free text, vocabulary and all.