Skip to content
AI360XpertBeta

Model Monitoring

A deployed model fails silently, returning confident wrong answers while accuracy labels lag by days. Monitoring watches proxy metrics that move first and pages you before users notice.

Refusal rate climbs from 2.1 to 4.8 percent, up 129 percent against a 30 percent rule, while P99 latency crosses its 3000 millisecond line days before accuracy labels arrive
Refusal rate climbs from 2.1 to 4.8 percent, up 129 percent against a 30 percent rule, while P99 latency crosses its 3000 millisecond line days before accuracy labels arrive

Why Does This Exist?

A web server fails loudly: it returns a 500 and someone gets paged. A model fails silently: it returns a 200 with a confidently wrong answer, and keeps doing that for three weeks before anyone notices. By then the damage is measured in lost users, and the cause is buried under weeks of data, prompt, and traffic changes.

The hard part is that true accuracy is almost never available in real time. Ground truth labels arrive days late or never. So monitoring watches proxies instead: refusal rate, output length spread, online quality scores, input shapes, latency, error counts, and spend. These move first. A refusal rate that doubles this week is smoke; the accuracy fire gets confirmed much later.

This page covers the watch: which metric families to track, how to set alert lines from your own baseline, and one worked week you can check by hand. The math that compares training and live input windows lives in data drift detection. The case where inputs look identical but the right answer moved lives in concept drift. The shelf that a flagged model rolls back to lives in the model registry.

Think of It Like This

A smoke detector, not a fire inspector

A smoke detector does not measure how much of the house has burned. It watches one proxy, smoke particles in the air, and beeps minutes before flames are visible. You pick its spot, learn its moods, and trust an early beep over a late inspection.

Refusal rate is the smoke. The 30 percent relative rule is the beep threshold. The two quiet weeks that set the 2.1 percent baseline are learning the kitchen's normal cooking smoke. Accuracy labels are the fire inspector, precise and days late.

Where it stops: a detector cannot tell burnt toast from a real fire, and it never puts anything out. Proxies false alarm the same way, and an alert pages a human or a rollback instead of fixing the model itself.

How It Actually Works

Track four families, one metric each to start. Output quality: refusal rate or an online judge score. Input signal: query length spread or the share of a new topic cluster. Infrastructure: P99 latency and error rate by type. Cost: spend per thousand requests. Five small lines beat fifty ignored ones.

No line gets a copied threshold. Watch everything quietly for two weeks, record the baseline mean, then alert on movement from that mean. Two rule shapes cover nearly everything. A static rule guards a hard promise: page when P99 latency passes 3000 milliseconds. A relative rule guards a drifting habit: page when a metric moves more than 30 percent from its 7-day rolling mean.

Worked example: the week refusals doubled

A support bot logs refusals per thousand requests. Baseline week: 22, 19, 24, 21, 20, 23, 18. Sum is 147, so the mean is 147 divided by 7, which is 21.0 per thousand, or 2.1 percent. This week: 48, 52, 44, 50, 47, 53, 42. Sum is 336, so the mean is 336 divided by 7, which is 48.0 per thousand, or 4.8 percent.

Relative move is (48.0 minus 21.0) divided by 21.0, times 100: 27 divided by 21 is 1.2857, so about 128.6 percent, call it 129 percent. The relative rule fires at anything past 30 percent, so this pages loudly. Same week, P99 latency moves from 1200 to 3100 milliseconds against the static 3000 line, so that pages too. The judge score slips from 8.4 to 7.1 against a 7.5 floor, and spend per thousand requests rises from 42 to 61 cents against a 55 cent line. Two independent families agreeing is what turns a page into a rollback instead of a shrug. True accuracy for this week arrives eleven days later and confirms the drop, which is exactly why the proxies exist.

Code

The point is the check: rebuild both weekly means from the worked example and confirm the 129 percent lift trips the 30 percent rule.

baseline: list[int] = [22, 19, 24, 21, 20, 23, 18]current: list[int] = [48, 52, 44, 50, 47, 53, 42]
mean_base: float = sum(baseline) / len(baseline)mean_cur: float = sum(current) / len(current)lift_pct: float = (mean_cur - mean_base) / mean_base * 100.0
print(f"{mean_base:.1f} {mean_cur:.1f} {lift_pct:.1f}")  # -> 21.0 48.0 128.6print(lift_pct > 30.0)  # -> True

The printout matches the hand computation: 2.1 percent baseline, 4.8 percent current, 128.6 percent lift, rule tripped. Change any current-day count by 5 and rerun to feel how jumpy small denominators are, which is why the rule compares weekly means rather than single days.

Watch Out For

Copying someone else's threshold

A blog post says alert at 5 percent refusals, so a team sets 5 percent on a bot whose normal is 2.1 percent. Nothing fires until the problem is enormous, and then everything fires at once. Another team copies a 1 percent line onto a bot whose normal is 3 percent and gets paged every night until they mute the channel.

Symptom: alerts that either never fire or fire constantly from the first week, with no quiet period that matched real incidents. Fix: run two weeks with no alert lines, record your own mean, then set the relative line at 30 percent above it and the static lines only from your own SLA promises.

Quoting a proxy as accuracy

The dashboard shows refusals at 4.8 percent, up from 2.1, and the status update says accuracy fell 2.7 points. It did not. Refusals moved; accuracy is still unknown until labels arrive. Decisions made on that sentence, like retraining on this week's traffic, rest on a number nobody measured.

Symptom: incident reviews that cite proxy moves as accuracy moves, and retrains launched before labels land. Fix: label every dashboard card as proxy or ground truth, and let proxies trigger investigation and rollback only. The words for the two drift shapes behind most proxy moves are data drift and concept drift.

The Quick Version

  • Deployed models fail silently with confident wrong answers, so monitoring watches proxies that move before accuracy labels arrive.
  • Our bot week moves refusals from 21.0 to 48.0 per thousand, a 128.6 percent lift that trips the 30 percent relative rule.
  • P99 latency crossing its 3000 millisecond static line in the same week is the second vote that turns a page into a rollback.
  • Set every line from your own two-week baseline, since copied thresholds either sleep through fires or page nightly.
  • Proxies trigger investigation, never an accuracy claim, because the true score stays unknown until labels land.