Structured Pruning Channels
Structured pruning removes whole channels and filters, so the smaller model still runs as dense math and speeds up on real hardware.
Why Does This Exist?
A ResNet 50 that scores 76 percent top 1 is too slow for a checkout camera at 15 watts. Model pruning can shrink it, but random weight zeros need sparse kernels to speed up. Structured pruning exists to cut whole channels instead: the network stays dense, smaller, and faster on any GPU or DSP with no special kernels.
General compression lives elsewhere. The MLOps overview places pruning among serving options. This page is the vision routine: rank channels, drop, retrain briefly, and measure wall time, not just parameter counts.
Basics first: a convolution with C out channels and C in channels costs proportional to both. Dropping one output channel removes its filters and one input channel downstream.
Think of It Like This
Closing checkout lanes, not scattered tills
A store with 16 lanes closes 6 whole lanes at quiet hours and staff rest. It does not remove every sixth item from every belt. Whole lane closure keeps flow simple. Structured pruning closes whole channels. Unstructured pruning removes scattered weights and needs special routing to see speed. The analogy stops at retraining: closed lanes reopen unchanged, while pruned nets must heal by retraining.
How It Actually Works
Train normally, then score each channel by its batchnorm scale magnitude or L1 filter norm. Remove the lowest fraction globally or per layer with a floor to protect early edges. Fine tune 5 to 10 epochs at low learning rate to let survivors absorb load. Repeat once if needed. Measure latency on target hardware because FLOPs and time diverge on depthwise layers.
Uniform cuts break the assumption. Early layers carry edges for all classes. Systems protect the first block and prune late stages harder, then verify per class accuracy, not just top line mean.
Worked channel cut
One stage holds 256 channels. Scales rank, lowest 96 drop. Remaining are 160. Channel fraction kept is 160 divided by 256 which equals 0.625. FLOPs fall about 37.5 percent in that stage. After 8 epochs of healing, top 1 moves 76.2 to 75.6, a 0.6 point cost for a 1.6 times stage speedup measured on device.
Watch Out For
Counting parameters instead of timing hardware
A 40 percent parameter cut on 1 by 1 convolutions can yield only 12 percent time save on memory bound phones. The symptom is a small model that still misses frame budget. Fix it by profiling latency per layer and by pruning where time actually spends.
The Quick Version
- Structured pruning drops whole channels to keep dense fast math.
- Rank by batchnorm scale, cut low, retrain briefly.
- Our cut kept 160 of 256 channels, FLOPs down 37.5 percent.
- Protect early layers and verify per class drops.
- Time on device, because parameter counts mislead.