ENAS Shares Weights to Search Fast

ENAS trains one supernet where child models share weights, so controller search costs hours not thousands of days.

Shared supernet plus controller cuts search to 16 hours

Why Does This Exist?

NASNet trains $20{,}000$ child nets from scratch for $2000$ days. Sharing nothing wastes nearly all compute.

ENAS shares all. It precedes darts gradients, fits efficient architectures, and searches image classification cells.

Think of It Like This

One kitchen serving many menus

Separate kitchens per menu waste stoves. One shared kitchen with rotating menus tests recipes fast with shared tools.

Supernet is the kitchen, controller picks menus. Where it stops: shared tools bias menus, just as shared weights bias rankings.

How It Actually Works

Build DAG supernet where each edge holds shared op weights. RNN controller samples child $a$ with prob $\pi(a)$ . Train shared $w$ on sampled child loss, then train controller by REINFORCE on val accuracy reward.

Worked example

Controller picks conv with prob $0.6$ reward $0.80$ versus pool $0.4$ reward $0.70$ . Expected reward $0.6 \times 0.80 + 0.4 \times 0.70 = 0.48 + 0.28 = 0.76$ . Gradient raises conv logit by $(0.80-0.76) = 0.04$ times. Search $16$ hours versus $2000$ days ( $48{,}000$ hours), $3000$ times cheaper.

Watch Out For

Trusting shared weight rankings directly

Shared scores misorder standalones. Symptom: top shared cell flops alone. Fix: retrain top few from scratch and pick by standalone val.

The Quick Version

ENAS shares weights in one supernet across all children.
RNN controller samples by REINFORCE on val reward.
Fixture expected reward $0.76$ guides controller updates.
Search $16$ hours replaces $48{,}000$ , 3000 times cheaper.
Retrain top cells alone before final pick.