Jetson Vision Deployment

Why Does This Exist?

A store with 40 cameras cannot push 40 times 4 Mbps to cloud and still alert in under a second. Bandwidth bills climb and a cut cable blinds the aisle. Jetson exists to run the detector where the pixels land: on a 7 to 25 watt box beside the cameras.

The box is an NVIDIA Orin Nano Super module: 1024 Ampere CUDA cores plus 32 tensor cores, 8 GB LPDDR5 at 102 GB per second, 67 sparse INT8 TOPS in MAXN SUPER mode at 25 watts, 33 dense. Production JetPack 6.2 unlocks the Super clocks. Price is about $249 for the developer kit, so a 40 camera site can afford one box per 8 cameras.

This page is the Jetson pick and bring up before runtime tuning. Edge deployment splits camera, gateway, and cloud work. TensorRT compiles the engine. Inference optimization sets the frame budget first.

Basics first: an engine binds to chip, batch, and input size. A Jetson Orin engine will not run on a server T4. Calibration sets INT8 scales from about 500 representative images.

Think of It Like This

How It Actually Works

Pick the mode first. Orin Nano 8 GB offers 7 watt, 15 watt, 25 watt, and uncapped MAXN SUPER. Vision sites standardize on 15 watts: about 67 sparse TOPS headroom without ceiling heat sink throttling. Set it with sudo nvpmodel -m 2 and lock clocks with sudo jetson_clocks during measurement, or lab numbers will not repeat in ceilings.

Input and output are a contract. Input is NCHW RGB at a fixed size, say 640 by 640, FP16 or INT8. Output is boxes plus scores plus classes. Export ONNX at that exact size, build the TensorRT engine on the Jetson itself, and keep batch 1 for cameras. Dynamic shapes cost about 15 percent unless explicit profiles are built.

Power decides placement. Eight cameras at 1080p and 15 fps need about 4 TOPS for a nano detector. One Orin Nano at 15 watts holds that with headroom for tracking. Two hundred cameras need 25 such boxes, not four cloud A100s plus 800 Mbps uplink.

Worked frame math

Baseline FP16 at 45 ms is 22.2 fps since 1000 divided by 45 equals 22.22. INT8 TensorRT at 18 ms is 55.6 fps since 1000 divided by 18 equals 55.56, a 2.5 times gain since 45 divided by 18 equals 2.5. At 15 watts that is 3.70 fps per watt since 55.56 divided by 15 equals 3.70. Model file falls 12 MB FP32 to 3 MB INT8, 4 times smaller since 12 divided by 3 equals 4.

Code

The point is the budget check before buying boxes. It recomputes fps, gain, and fps per watt from two latency numbers.

1import numpy as np2
3
4def fps(latency_ms: np.ndarray) -> np.ndarray:5    return 1000.0 / latency_ms  # frames per second from ms per frame6
7
8def fps_per_watt(fps_val: float, watts: float) -> float:9    return fps_val / watts  # efficiency the thermal design must hold10
11
12lat = np.array([45.0, 18.0])13print(fps(lat))  # -> [22.22222222 55.55555556]14print(fps(lat)[1] / fps(lat)[0])  # -> 2.515print(fps_per_watt(float(fps(lat)[1]), 15.0))  # -> 3.7037037037037033

Use 15 watt locked clocks for the 18 ms number. Uncapped MAXN flatters it by about 20 percent and then throttles in hot ceilings.

Watch Out For

The Quick Version

• Jetson Orin Nano Super gives 67 sparse TOPS with 8 GB at 7 to 25 watts for camera side inference.
• Standardize on 15 watts with locked clocks or field fps will not match lab fps.
• Our frame fell 45 ms to 18 ms INT8, 22.2 to 55.6 fps, 2.5 times faster.
• That is 3.70 fps per watt at 15 watts, the number ceilings must hold.
• Build the TensorRT engine on the deployment chip itself, since engines do not port across chips.

What to Read Next

• Edge Embedded Deployment
• TensorRT Vision Compile
• Inference Optimization Vision