Once-for-All Trains One Net for Many

Once-for-All trains a supernet supporting many widths and depths, so phones pick subnets instantly with no retraining.

Elastic training once gives instant subnets for any phone budget

Why Does This Exist?

Per-phone search repeats training per latency band. Ten phones mean ten searches.

OFA trains once. It follows proxylessnas hardware care, serves efficient architectures, and ships mobilenet class fleets.

Think of It Like This

A master key cut down for many locks

You forge one large key blank, then file copies to fit small locks in seconds without reforging.

Supernet is the blank, subnets are filings. Where it stops: filings cannot exceed blanks, just as subnets cannot beat supernet capacity.

How It Actually Works

Train max net $8$ M, then progressively shrink to $6$ M and $4$ M with distillation from max. Support elastic depth $2$ - $4$ , width $3$ - $6$ , kernels $3$ - $7$ . Build latency tables per phone, search subnets by predictor with no training.

Worked example

Max $8.0$ M at $80$ percent. Sub $6.0$ M keeps $79.2$ percent ( $0.8$ drop for $25$ percent fewer weights: $(8-6)/8 = 0.25$ ). Sub $4.0$ M keeps $77$ percent ( $3$ drop for half weights). Lookup picks $6$ M for $15$ ms budgets and $4$ M for $10$ ms in seconds.

Watch Out For

Sampling small subnets too early in shrinking

Early small samples starve before large stabilizes. Symptom: tiny subnets lag 5 points. Fix: follow max to middle to small progressive order.

The Quick Version

Once-for-All trains one elastic supernet for many phone budgets.
Progressive shrinking distills max to middle to small.
Fixture $8$ M to $6$ M loses $0.8$ points for $25$ percent fewer weights.
Latency tables pick subnets in seconds with no retrain.
Keep max to small order or tiny nets stall.