ML Testing Strategies

Why Does This Exist?

A pipeline passes every unit test and still ships a broken model. The scaling function is correct, the pipeline stages connect, and the deploy succeeds. Then support tickets arrive: the model rejects a slice of users it used to accept. Nothing in the code changed, because the failure was never in the code. The incoming age field started arriving in months instead of years, or accuracy on one minority slice quietly fell to 0.60 while the global number stayed a comfortable 0.95.

ML testing exists because an ML system is three things that break independently: code, data, and learned behavior. Code tests cannot see a shifted column. Data checks cannot see a biased decision boundary. Aggregate accuracy cannot see a failing slice. Each layer needs its own gate, and a release passes only when all three pass. If the lifecycle is new, read the MLOps overview first.

This page covers pre-deploy gates: pipeline unit tests, input validation, and behavior checks. Watching those same distributions after release is data quality monitoring and model monitoring. Catching shifts in live traffic is model drift detection.

Basics first: a slice is a named subset you grade separately, like applicants under 25 or transactions above 10,000. Global accuracy is the average over everyone, which lets a large healthy majority hide a small failing slice. Everything below grades slices alongside the global number.

Think of It Like This

How It Actually Works

Three gates run in order on every candidate release. Call the pipeline code $C$, which means transforms plus wiring. Call the input batch $D$, which means one dataframe with a schema and distributions. Call the trained model $M$, which means weights plus a decision rule. Gate each one before it can hide behind the next.

Gate one: code behaves on fixed fixtures

Unit-test transforms with hand-computed fixtures. A scaler fitted on training data must map 42.32 to the same output every run, and an unknown category must raise rather than silently become zero. Integration tests push 50 mock rows through ingest to featurize to train to export and assert shapes match: 50 in, 50 out, no null prediction. These tests run in seconds and catch wiring breaks before any data question arises. Templates that provision this pipeline get the same treatment as infrastructure as code.

Gate two: data matches the contract

Validate schema first: expected columns, types, and bounds. Age stays within 18 to 120, income stays at or above 0, status stays inside the allowed set. Then compare distributions against the training baseline: means within tolerance, null rate below ceiling, category shares near history. A batch with 22 percent nulls in a column that trained at 1 percent never reaches training, no matter how green the code gate is.

Gate three: behavior holds on slices and probes

Grade accuracy per slice, not just globally. Add invariance probes: swapping a name that should not matter must not flip the decision. Add directional probes: raising income while holding everything else fixed should not lower approval probability. Block the release when any probe or any guarded slice fails, even when the global number passes.

Worked example

A candidate scores 950 correct of 1,000 rows, so global accuracy is $950 / 1,000 = 0.95$. Split it: the majority slice scores 902 of 920, which is $902 / 920 = 0.9804$, while the minority slice of 80 rows scores 48 of 80, which is $48 / 80 = 0.60$. Check the sum: $902 + 48 = 950$ and $920 + 80 = 1,000$, so the global 0.95 is exactly the blend that hides the failing 0.60. The code gate passed, the schema gate passed, and only the slice gate blocks this release.

Code

The point is the hidden slice, computed with stdlib only. Same totals as the worked example, plus a schema check and two behavior probes.

1import statistics2
3correct_major, n_major = 902, 9204correct_minor, n_minor = 48, 805global_acc = (correct_major + correct_minor) / (n_major + n_minor)6print(round(global_acc, 4))  # -> 0.957print(round(correct_major / n_major, 4))  # -> 0.98048print(round(correct_minor / n_minor, 4))  # -> 0.69
10
11def schema_ok(rows: list[dict]) -> bool:12    for r in rows:13        if not (18 <= r["age"] <= 120):14            return False15        if r["income"] < 0:16            return False17        if r["status"] not in {"active", "inactive"}:18            return False19    return True20
21
22batch = [23    {"age": 34, "income": 52.0, "status": "active"},24    {"age": 17, "income": 40.0, "status": "active"},25]26print(schema_ok(batch))  # -> False27
28train_mean = statistics.mean([41.2, 42.5, 43.1, 42.0, 42.8])29live_mean = statistics.mean([41.9, 42.3, 42.6, 42.1, 42.4])30print(round(train_mean, 2), round(live_mean, 2))  # -> 42.32 42.26

Global 0.95 passes, minority 0.60 fails, one underage row fails the schema, and the two means 42.32 and 42.26 agree. Three gates, three verdicts, one blocked release.

Watch Out For

The Quick Version

• Test code, data, and behavior separately, because each fails in ways the others cannot see.
• Code gates use fixed fixtures and mock batches to catch wiring and transform breaks.
• Data gates enforce schema bounds and compare means, null rates, and category shares to baseline.
• Behavior gates grade guarded slices plus invariance and directional probes on the trained model.
• A 0.95 global score can hide a 0.60 slice when 902 of 920 plus 48 of 80 blend into 950 of 1,000.
• Block the release when any gate fails, even when the global number passes.
• Leaked splits make every gate lie, so assert zero ID overlap before trusting a score.

What to Read Next

• Model Drift Detection
• Data Quality Monitoring
• Model Monitoring Platform: Drift to Alert