YOLO Architecture

YOLO frames object detection as a single regression problem, letting one network look at an image once and predict boxes and classes together.

One grid over the image predicts boxes and classes in a single pass, and NMS keeps the best box where predictions overlap.

Why Does This Exist?

Before YOLO, detection was a slow pipeline. R-CNN methods generated thousands of region proposals, ran a classifier on each one, then cleaned up duplicates, which was accurate but far from real-time. Self-driving cars, robotics, and live video cannot wait for thousands of classifier runs per frame.

YOLO reframed the whole task as one regression problem. A single convolutional network sees the image once and directly predicts every bounding box plus its class probabilities in one forward pass. This page covers the grid rule, the output tensor, the worked NMS cleanup, and the small-object price of that speed. Proposals-based two-stage craft is the alternative, and the cleanup step has its own page under non-maximum suppression.

Think of It Like This

The efficient mail sorter

A sorter watches a fast conveyor belt with a laser grid drawn across it. Instead of pausing the belt to inspect every inch with a magnifying glass, the sorter takes one glance at the whole grid and calls out per zone: a medium box centered here, a small envelope centered there.

The glance is the single forward pass, each grid zone is responsible only for objects centered inside it, and overlapping shouts about the same parcel get merged into one. The analogy stops at learning: the sorter trained for years, while YOLO learns its glance from labeled boxes by gradient descent.

How It Actually Works

The grid responsibility rule

YOLO divides the image into an $S \times S$ grid, $7 \times 7$ in the original paper. The core rule is that if the center of an object falls into a grid cell, that cell is responsible for detecting it. Each cell predicts $B$ bounding boxes, $B = 2$ originally, with five numbers each: center $x$ and $y$ relative to the cell bounds, width $w$ and height $h$ relative to the whole image, and a confidence score $P(\text{object}) \times \text{IoU}$ that blends whether anything is there with how good the box is.

One set of classes per cell

Each cell also predicts $C$ conditional class probabilities, $P(\text{class} \mid \text{object})$ , one set per cell regardless of $B$ . With $S = 7$ , $B = 2$ , $C = 20$ , the output tensor is $7 \times 7 \times (2 \times 5 + 20) = 7 \times 7 \times 30$ , which is $49 \times 30 = 1{,}470$ numbers from a single pass.

Worked NMS example

Neighboring cells often fire on the same object, so YOLO sorts boxes by final score, $P(\text{object}) \times P(\text{class} \mid \text{object})$ , and suppresses any box overlapping a kept one above the IoU threshold. Take box A $[10, 10, 50, 50]$ at score $0.9$ and box B $[12, 12, 52, 52]$ at score $0.7$ . Both are $40 \times 40$ , intersection is $38 \times 38 = 1{,}444$ , union is $1{,}600 + 1{,}600 - 1{,}444 = 1{,}756$ , so $\text{IoU} = 1{,}444 / 1{,}756 \approx 0.822$ . Above a $0.5$ threshold, B is suppressed and A survives. A distant box $[100, 100, 140, 140]$ at $0.8$ overlaps neither and is kept.

Code

IoU plus greedy NMS on the worked boxes:

1def iou(a, b):2    ix0, iy0 = max(a[0], b[0]), max(a[1], b[1])3    ix1, iy1 = min(a[2], b[2]), min(a[3], b[3])4    inter = max(0, ix1 - ix0) * max(0, iy1 - iy0)5    ua = (a[2] - a[0]) * (a[3] - a[1])6    ub = (b[2] - b[0]) * (b[3] - b[1])7    return inter / (ua + ub - inter)8
9boxes = [((10, 10, 50, 50), 0.9), ((12, 12, 52, 52), 0.7), ((100, 100, 140, 140), 0.8)]10print(round(iou(boxes[0][0], boxes[1][0]), 4))11# -> 0.822312
13kept = []14for b, s in sorted(boxes, key=lambda t: -t[1]):15    if all(iou(b, k) < 0.5 for k, _ in kept):16        kept.append((b, s))17print(kept)18# -> [((10, 10, 50, 50), 0.9), ((100, 100, 140, 140), 0.8)]

The $0.822$ overlap kills the weaker duplicate while the distant box survives, matching the hand computation.

Watch Out For

The small object crowd problem

Each cell predicts a fixed number of boxes and one class set, so a flock of small birds inside a single cell cannot all be predicted. The symptom is clustered small objects detected as one or missed entirely. Later versions add anchor boxes and feature pyramids to soften this, but dense crowds remain the weak spot.

Thresholding confidence instead of the final score

Confidence is $P(\text{object}) \times \text{IoU}$ , not the class probability, so thresholding it alone keeps well-localized boxes of the wrong class. The symptom is confident mislabeled boxes surviving filtering. Always multiply by the conditional class probability first and threshold the product.

The Quick Version

YOLO reframes detection as one regression: one network, one forward pass, all boxes and classes.
Each cell owns objects centered inside it and predicts $B$ boxes plus one class distribution.
Original output is $7 \times 7 \times 30$ , which is $1{,}470$ numbers per image.
Worked NMS: boxes at IoU $0.822$ collapse to the stronger one above a $0.5$ threshold.
Real-time speed costs small-object accuracy where crowds share one cell.