Automated Feature Engineering

Why Does This Exist?

Nine related tables in a warehouse. Writing features by hand costs a day per idea, most of it spent working out which join gives the right grain. A library reads your schema, walks the relationships, and hands back 900 columns before lunch.

Real win, narrower than it looks. Automation is good at enumeration and bad at judgement, so it wins where the candidate space is large and mechanical — many tables, many plausible aggregates per relationship — and loses where the useful column needs knowing what the business does. No generator was producing days_until_contract_renewal unless told renewal dates exist and matter.

Let the machine enumerate. Keep the judgement, especially about time.

Think of It Like This

How It Actually Works

Deep feature synthesis

You hand the tool a set of tables and the keys joining them, and it stacks two kinds of primitive. Aggregation primitives cross a one-to-many relationship and collapse many child rows into one number per parent: count, sum, mean, max, std, num_unique, trend. Transform primitives stay inside a row and rewrite a value: day-of-week, time-since, percentile, cumulative sum.

Depth is how many times it will stack them. Depth 1 gives "number of orders per customer". Depth 2 gives "mean of the number of items per order, per customer" — nonsense until you notice it's basket size, and no analyst was going to name it. That composition is the value proposition.

Four ways it goes wrong

Leakage at scale, which matters more than everything else here. An aggregate has no notion of prediction time unless you give it one per row, so the default output covers every child row that exists today. A MEAN(orders.amount) on a January training row quietly includes November. Nothing is null, nothing errors, and the column ranks first on importance (data leakage).

Overfitting the selection. A thousand candidates scored against one validation set will find winners in pure noise, and the bigger the pool, the more certain that becomes (multiple comparisons).

Columns nobody can explain. STD(orders.MEAN(items.unit_price)) may well predict churn. Try defending it to a regulator, or to the analyst judging a drift alert.

Serving cost. Every column you keep must be recomputable per request, at prediction latency, from data that exists then — a depth-3 aggregate is a multi-table join under a latency budget (online and offline skew).

The automation that actually won

On unstructured input, what replaced hand-built features wasn't a primitive stacker. It was representation learning: a pretrained encoder turning text, an image or audio into a dense vector, learned from far more data than you have. That's automated feature engineering that works, and why this page hands off to vector embeddings. Structured tables never got an equivalent.

Worked example

Three tables in a chain: customers, orders, order_items, four numeric columns each. Six aggregation primitives, five needing a column and one — count — that doesn't.

One hop, order_items into orders: $5 \times 4 + 1 = 21$ new columns. Two hops: orders now carries its own 4 plus those 21, so 25 things to aggregate: $5 \times 25 + 1 = 126$ columns land on a customer row. Three hops: $5 \times (4 + 126) + 1 = 651$.

Two more primitives makes depth 3 into 1,653, and a second child relationship roughly doubles it again. Almost all noise, which is the point: a generator is half a system, and the other half is feature selection with an evaluation you trust.

Show Me the Code

1import numpy as np2
3AGGS = 6   # count, sum, mean, max, std, num_unique4OWN = 4    # numeric columns in every table5
6
7def candidates(depth: int, aggs: int = AGGS, own: int = OWN) -> np.ndarray:8    """Columns landing on the parent row after each extra hop up a chain of tables."""9    counts = np.zeros(depth, dtype=int)10    available = own11    for level in range(depth):12        counts[level] = (aggs - 1) * available + 1  # count needs no column, the five others do13        available = own + counts[level]14    return counts15
16print(candidates(3))          # -> [ 21 126 651]17print(candidates(3, aggs=8))  # -> [  29  232 1653]

available = own + counts[level] is where it runs away: last level's output is this level's input, so growth is multiplicative in depth.

Watch Out For

The Quick Version

• Automation is good at enumeration and bad at judgement: it wins on many related tables, loses where the column needs domain knowledge.
• Deep feature synthesis stacks aggregation primitives across relationships and transform primitives within a row, to a chosen depth. At depth 2 that gives "mean of the number of items per order, per customer".
• The arithmetic: 3 tables, 4 numeric columns, 6 primitives gives 21 candidates at one hop, 126 at two, 651 at three. Almost all noise, so a generator is half a system and the other half is selection.
• Leakage at scale matters most: without a per-row cutoff, every aggregate reads the whole child table, future included.
• The other three: overfitting the selection, columns nobody can explain, serving cost at prediction latency.
• On unstructured data the automation that won is representation learning, not primitive stacking.

What to Read Next

• Feature Construction is the hand-written version, and the as-of rule this automates.
• Feature Selection is the mandatory second half of any generator.
• Data Leakage is the failure a missing cutoff causes.
• SQL for Feature Extraction is how an as-of aggregate gets written.
• Vector Embeddings is the automation that worked on unstructured input.
• Definitions worth a look: Feature, Feature Matrix, and Data Leakage.