Feature Construction

This is where you add what the table doesn't already know. A ratio, a lag, a count in a window, and every one of them computed as of prediction time.

Five orders on a timeline become three constructed columns, and the prediction-time line decides which events may contribute: the 300 order is in the table but outside the 1 March features.

Why Does This Exist?

Scaling a column has roughly one right answer. Encoding one has about four. Construction has none until somebody knows something about the problem, which is why it holds the most headroom and the least automation.

A model combines your columns only in the ways its own family permits. A linear model adds them up. A tree cuts them one at a time, along axes. Neither divides two columns by each other, and neither knows that 03:00 on a Tuesday means something different from 13:00 on a Saturday. That knowledge lives in your head, a domain expert's, or a table nobody joined yet. Construction writes it into the feature matrix as a column. It is the only step that adds information rather than rearranging it, and on tabular problems it is usually where the winning margin comes from.

Think of It Like This

Firewood, priced by the honest kilo

Two merchants sell you a trailer of firewood for the same price. One load was split last winter and sat covered for a year. The other came down last week and is still green. Same trailer, same price, and one of them is nearly twice the heat.

Nobody printed heat on the ticket. You divide the price by the dry weight, write down price per seasoned kilo, and suddenly there is no argument. The number you reason with never existed until you made it, and making it took knowing which divisor matters.

Where it stops: firewood arithmetic is exact and the divisor is visible. A constructed feature is a bet about what matters, placed before the model runs. Price per kilo is true whether or not anyone buys. A ratio you invent is only useful if the target cares about it, and a clever column the target ignores is noise with a good story.

How It Actually Works

The moves worth knowing

Nearly every good tabular feature is one of about eight shapes.

Ratios and rates. Price per square metre, clicks per impression, debt to income. Dividing by size is the highest-yield move on tabular data.
Differences and deltas. Promised date minus actual date, current reading minus the rolling mean.
Date and time parts. Hour, day of week, is-weekend, days since signup. A raw timestamp only ever grows, so any split on it means "before or after this date".
Aggregates over a group. Orders per customer, mean basket value, distinct devices seen.
Lags and windows. Yesterday's value, the 7-day rolling mean. A lag feature is how a model reaches into the past.
Counts in a trailing window. Failed payments in the last 30 days, logins in the last 24 hours. Heavy hitters in fraud and churn.
Flags for a business rule. is_first_order, exceeds_credit_limit. Often what a domain expert names first.
Interaction terms. The product of two columns. An interaction feature does the multiplication a linear model's form forbids.

A ratio costs you one line and costs a tree a staircase of dozens of splits that only approximates it.

As of when

One sentence separates a feature from a bug. Every aggregate and every window is computed as of the moment you would actually have to predict.

Not as of today, and not as of whenever the export ran. As of the prediction timestamp for that row, so one customer's mean_order_value differs between a January row and a June row. If it does not, you built something that cannot exist in production.

Warehouses make the wrong version easier to write: GROUP BY customer_id over the whole table, joined onto a January row, looks tidy. The right version recomputes per row against only the events before that row's timestamp, which is what an as-of join does. See SQL for feature extraction. The split obeys the same clock, so a time dimension means a temporal cut: train, validation and test splits.

Worked example

One customer, five orders, and a churn model scoring them on 1 March 2026: 14 Jan 40, 6 Feb 55, 19 Feb 25, 24 Feb 60, 7 Mar 300. That last order sits on the wrong side of the line. It is in the table you build features from today, and it did not exist on 1 March.

days_since_last_order is 1 March minus 24 February, so 5. orders_in_prior_30d counts back to 30 January, catching 6 Feb, 19 Feb and 24 Feb, so 3. mean_order_value is $(40 + 55 + 25 + 60) / 4 = 45.00$ .

Build that last one over every row instead: $(40 + 55 + 25 + 60 + 300) / 5 = 96.00$ . Same column name, more than double the value, and the extra came from an order placed six days after the prediction. Nothing is null, nothing errors, and your validation score goes up.

Code

The whole discipline lives on the known line. Drop it and the leak walks in.

1import numpy as np2
3stamps = np.array(["2026-01-14", "2026-02-06", "2026-02-19", "2026-02-24", "2026-03-07"], dtype="datetime64[D]")4amounts = np.array([40.0, 55.0, 25.0, 60.0, 300.0])5
6
7def features_as_of(t: np.ndarray, amt: np.ndarray, cut: np.datetime64) -> dict[str, float]:8    known = t < cut9    days = (cut - t[known]) / np.timedelta64(1, "D")10    return {"days_since_last": float(days.min()), "orders_prior_30d": float((days <= 30).sum()),11            "mean_order_value": float(amt[known].mean())}12
13
14feats = features_as_of(stamps, amounts, np.datetime64("2026-03-01"))15print(feats)  # -> {'days_since_last': 5.0, 'orders_prior_30d': 3.0, 'mean_order_value': 45.0}16print(float(amounts.mean()))  # -> 96.0

Without known, the mean goes to 96.0, the count picks up a future order, and days_since_last turns into -6, the only visible symptom.

Watch Out For

An aggregate that quietly averages the future

mean_order_value computed with one GROUP BY over the whole history, then joined onto every training row. No null, no error, no column name that sounds suspicious.

A row dated 12 January carries a mean built partly from orders in March and November. The model learns that customers with a high lifetime mean do not churn, which is true and useless, because the high mean is partly caused by not churning. In production the column is computed from data that exists, the distribution moves, and your best feature stops working.

The tell is a feature that ranks first on importance, costs nothing when removed under a random split, and costs a lot under a time-ordered backtest. Believe the backtest, then fix it structurally: every aggregate as an as-of value keyed on the row's timestamp, validated on a temporal split. See data leakage.

Generating a thousand ratios and shipping all of them

Forty numeric columns give 780 pairs, so 2,340 candidates once you divide, multiply and subtract each one. Keep the top 200 by importance, watch validation land a shade above baseline, and ship.

Here is what went wrong. With 2,340 candidates and a few thousand rows, some pure-noise columns score well by luck. That is what selection does once the pool is big enough. They survive, and the model gets worse while its feature list looks richer.

Generating candidates is fine. Selecting from them with the data you report on is not. Nest selection inside cross-validation folds, keep a final untouched split, and expect most candidates to die. Automated feature engineering goes further down this road.

The Quick Version

Construction is the only step that adds information. Encoding, scaling and selection rearrange what is there.
Eight shapes cover most of it: ratios, differences, date parts, group aggregates, lags and windows, trailing counts, rule flags, interactions.
Dividing by size is the highest-yield move on tabular data. A raw timestamp is nearly useless; its parts are strong.
Neither trees nor linear models divide one column by another.
Every aggregate and every window is computed as of the row's prediction time.
The same aggregate over all history is a leak with no null in it: 45.00 becomes 96.00, silently.
Generating thousands of candidates is fine. Selecting from them with the data you report on is not.