ProxylessNAS Searches Direct on Hardware

ProxylessNAS binarizes paths and measures phone latency, so search optimizes accuracy plus real device speed directly.

Binary gates plus phone ms loss find fast mobiles directly

Why Does This Exist?

Proxy search on small sets then transfers misses phone latency. FLOPs proxies misrank real speed.

Proxyless measures devices. It follows darts relaxation, extends enas efficiency, and outputs mobilenet class phones.

Think of It Like This

Tailoring suits on the wearer not a dummy

Dummy fittings miss shoulder slope. Fitting on the wearer costs more per try but needs no remake.

Proxy sets are dummies, target phones are wearers. Where it stops: wearer fittings cost per device, just as direct search repeats per phone.

How It Actually Works

Keep $N$ paths per block with binary gates $g$ sampling one or two paths per batch to fit memory. Loss $CE + \lambda \log Latency$ . Update $w$ and $\alpha$ alternately with straight-through gradients plus latency predictor.

Worked example

Paths A $10$ ms $78$ percent, B $6$ ms $76$ percent. Loss A $-\log 0.78 + 0.1 \log 10 = 0.248 + 0.230 = 0.478$ . Loss B $0.274 + 0.179 = 0.453$ . B wins despite $2$ points lower because $4$ ms saved outweighs. Search finds $74$ percent mobiles $3$ times faster than proxy stacks.

Watch Out For

Overweighting latency and collapsing to tiny nets

High $\lambda$ picks all skips. Symptom: very fast but useless accuracy. Fix: tune $\lambda$ to hit target ms bands and check accuracy floors.

The Quick Version

ProxylessNAS searches on target task plus measured latency directly.
Binary gates sample one path to fit memory.
Fixture loss $0.453$ beats $0.478$ by trading 2 points for 4 ms.
Finds phone nets 3 times faster than proxy transfer.
Tune latency weight to hold accuracy floors.