Skip to content
AI360Xpert
Beta
Core ML

Visual explainer

Learning Curves

Plot training error against held out error as data grows, and the gap tells you whether to buy labels or add muscle.

More labels cost real money, so first ask whether data is even the cure.
More labels cost real money, so first ask whether data is even the cure.

Five hundred new labels cost weeks of work, yet sometimes error refuses to move. Buying data before diagnosing burns budget on the wrong cure. You need a chart that names the disease first.

Two Lines

Plot training error and held out error together against data size.
Plot training error and held out error together against data size.

Draw training error beside held out error as data grows. One line shows memorising, the other shows generalising, and the space between them is the diagnosis. Learn to read that gap and you stop guessing.

Flat And High

A too simple model stalls at 0.34 and 0.36, and extra data changes nothing.
A too simple model stalls at 0.34 and 0.36, and extra data changes nothing.

A line cannot learn curves. Training stalls at 0.34, held out at 0.36, and tripling the data moves neither. The gap is a tiny 0.02, yet both lines sit far too high. More examples cannot teach flexibility.

Wide Gap

A memorizing model scores 0.08 at home but 0.29 away, a gap of 0.21.
A memorizing model scores 0.08 at home but 0.29 away, a gap of 0.21.

A giant model recites instead of understanding. It scores 0.08 on training and 0.29 on fresh data, leaving a 0.21 gap. Low at home, lost away: the signature of memorisation over pattern.

Data Closes Gaps

Tripling the data pulls held out error 0.29 down to 0.17.
Tripling the data pulls held out error 0.29 down to 0.17.

Same model, more examples, and the story heals. At 300 points training reads 0.12 and held out 0.17, narrowing the gap to 0.05. Memorisation starves when examples outnumber hiding spots.

Prescribe Right

A wide gap asks for data, matching high lines ask for a bigger model.
A wide gap asks for data, matching high lines ask for a bigger model.

Let the gap prescribe. Wide gap with low training error means buy data. Matching high lines mean buy muscle: layers, features, training time. Habit says always add data; the chart often disagrees.

Where It Breaks

Noisy labels set a floor near 0.25 that neither data nor size can cross.
Noisy labels set a floor near 0.25 that neither data nor size can cross.

Sometimes the labels themselves lie. With one in four tags wrong, both lines sink to 0.25 and rest there forever. Neither data nor muscle crosses a noise floor. Fix the labels before buying more of them.

The Quick Version

  • Labels cost money, cures vary.
  • Plot memorising beside generalising.
  • Flat high lines need bigger models.
  • Wide gaps need more examples.
  • Data dragged 0.29 down to 0.17.
  • The gap picks your prescription.
  • Noise floors stop both remedies.