Scaling Laws Vision Curves

Double data or weights and error falls by a steady fraction on a log plot. Straight lines plan budgets before big runs.

Vision error falls as a power law in data plus model size

Why Does This Exist?

Big runs cost millions, so guessing size wastes budget. Scaling laws fit error $E = a D^{-b} + c$ for data $D$ plus similar terms for params $P$ and compute $C$ . Straight lines on log plots in foundation models planning pick the mix.

This page covers empirical curves only. Training recipes live in sibling pages.

The one idea you need first.

Power law means $\log E$ falls linearly in $\log D$ with slope $-b$ near $0.3$ . Irreducible $c$ is the floor. Compute-optimal balances $D$ and $P$ together.

Think of It Like This

Farming yield versus seed plus water on log paper

Doubling seed lifts yield less each time, and doubling water does too. On log paper both trace straight lines, so a farmer splits coins between bags and pipes for top yield per coin.

Data is seed, params are water, and error is hunger. Where it stops: farms hit seasons, while models hit data quality walls.

How It Actually Works

Fit $E(D) = a D^{-0.3} + c$ on small runs, then extrapolate. Repeat for $P$ at fixed $D$ . Pick $(D,P)$ minimizing $E$ under $C \approx 6PD$ . Refit per task family since dense tasks slope less steeply than tags.

In plain words: $a$ sets level, $b$ sets slope, and $c$ sets the floor no scale beats.

Worked example

Errors $24.0, 19.5, 15.8$ percent at $10$ M, $40$ M, $160$ M photos. Ratios $0.812, 0.810$ per $4$ x data. Slope $b = -\log(0.81)/\log(4) = 0.21/1.386 = 0.15$ here, milder than $0.3$ SOTA mixes. Next $4$ x to $640$ M predicts $15.8 \times 0.81 = 12.8$ percent. Budgets follow the line, not hope.

Watch Out For

Extrapolating past data quality cliffs

Noisy web data breaks the line past some point and error stalls. Plans overshoot by $2$ x compute. Refit after cleaning and cap runs at the measured bend.

The Quick Version

Vision error follows power laws in data, params, compute.
Log plots turn curves into straight budget lines.
Slope near 0.3 guides joint scaling.
Floors from quality cap extrapolation.
Refit per task family before big spends.