TFLite Mobile Vision

TFLite shrinks detectors to phone sized files with full integer math, so Android cameras run offline at 30 ms.

A 12 MB detector becomes 3.1 MB INT8 and runs 34 ms on phones with delegates on

Why Does This Exist?

Stores use mid Android phones as scanners with no signal in aisles. A 12 MB FP32 model plus Python will not fit the workflow. TFLite exists as the mobile purpose: convert to a flat buffer, quantize to INT8, and run with NNAPI or GPU delegates. Input is a trained detector. Output is a .tflite file plus delegate choice.

Mobile follows. Mobile deployment handles app wiring. This page is the convert and delegate pass with size and latency proof.

Basics first: delegates route ops to DSP or GPU. Unsupported ops fall back to CPU and stall. The converter report lists which ops delegate cleanly.

Think of It Like This

A travel toothbrush kit

Home kit holds full bottles. Travel kit holds small tubes that fit a pouch and pass checks. TFLite is the travel kit: same paste, smaller tubes, pouch sized. The analogy stops at compute: small tubes brush the same, while INT8 math can shift tiny boxes unless calibrated.

How It Actually Works

Export with 320 fixed shape. Convert with optimize default and 400 representative shelf images for full integer mode. Enable NNAPI plus GPU delegate with CPU fallback. Test on two mid phones, not flagships. Measure cold start, warm 34 ms target, and tail p95 under load with camera running.

Calibration drift breaks the assumption. Converter tuned on bright labs fails on dim aisles. Systems calibrate on store light and verify small tag recall separately.

Worked mobile pack

FP32 weighs 12.0 MB. INT8 weighs 3.1 MB since 12 divided by 3.87 is about 3.1 with overhead. Latency moves 112 ms CPU to 34 ms with GPU delegate. Gain is 112 divided by 34 which equals 3.29. mAP moves 0.371 to 0.360, a 0.011 drop concentrated on sub 20 pixel tags.

Watch Out For

Ops that silently fall back to CPU

A custom NMS or odd resize runs on CPU while the rest uses GPU, adding 18 ms of copies per frame. The symptom is delegate on with no speedup. Fix it by reading the delegate report and by replacing flagged ops before ship.

The Quick Version

TFLite packs detectors to phone INT8 with delegates.
Convert with 400 store light frames for full integer mode.
Our pack fell 12 MB to 3.1 MB and 112 ms to 34 ms.
Accuracy held within 1.1 points except tiny tags.
Check delegate coverage or CPU fallback erases gains.