Visual explainer

Non-Maximum Suppression (NMS)

How greedy box selection removes duplicate detections, with exact overlap arithmetic and the cases it gets wrong.

1. Count Proposals, Not Objects

Three same-class proposals can describe only two objects.

Non-maximum suppression removes overlapping detection proposals using their scores and box overlap. It repeatedly keeps the highest-scoring remaining box, then removes sufficiently overlapping candidates. Here A and B describe one bottle; C describes another.

2. Compute The Overlap

Two four-by-four boxes shifted by one unit have IoU 0.60.

Use continuous coordinates: A is (0,0,4,4), B is (1,0,5,4). Both have area 16. Their shared area is 12; their union is 16 + 16 − 12 = 20. Intersection over union is 12/20 = 0.60.

3. Keep The Highest Score

Keep the highest score, then suppress a lower-scoring box above the IoU threshold.

A's 0.90 beats B's 0.80. At an IoU threshold of 0.50, B is suppressed because 0.60 > 0.50. This follows Torchvision's strict greater-than rule. The score ranks candidates; it does not measure their overlap.

4. Repeat On What Remains

Repeat with the best remaining candidate until the candidate set is empty.

C does not overlap A, so it remains available after the first round. The next round keeps C. Stop when the remaining list is empty. A lower score alone never causes suppression by NMS.

5. Read The Output

NMS removes a duplicate while retaining detections for two separate objects.

The output is a subset of the input boxes: A and C. Their coordinates stay unchanged. NMS does not refine boundaries, average coordinates, or verify that a retained proposal truly contains an object.

6. Change One Threshold

Raising the IoU threshold suppresses fewer boxes on the same input.

At threshold 0.70, this pair's 0.60 overlap no longer triggers suppression. Higher thresholds tolerate more overlap and can leave duplicates. Lower thresholds remove more overlapping boxes and can lose nearby objects. Choose using validation examples from the target scene.

7. Separate Predicted Classes

Class-wise NMS compares boxes only within the same predicted class.

A person and a bicycle may overlap legitimately. Class-wise NMS processes their candidate lists independently. Class-agnostic NMS compares across classes instead; that is a separate policy choice from the numerical IoU threshold.

8. Separate The Two Cutoffs

A score cutoff and an IoU threshold answer different questions.

A detector may filter low scores before NMS. With an illustrative score cutoff of 0.75, C's 0.70 is removed before any overlap comparison. Lowering the NMS threshold cannot recover a candidate already discarded by score filtering.

9. Greedy Is A Local Choice

Greedy NMS does not maximize the total score of retained boxes.

This new fixture places A between B and C. A overlaps each at 0.60, but B and C overlap only at one third. Choosing A first removes both. NMS does not search for a compatible subset with the largest summed score.

10. Where It Breaks

Overlapping boxes can belong to different real objects, so NMS can remove a true detection.

Two people can produce highly overlapping same-class boxes. The same rule that removes duplicates can now remove a true detection. Raising the threshold trades this error against surviving duplicates; overlap and score alone do not reveal object identity.

The Quick Version

Multiple proposals can describe the same object.
IoU divides intersection area by union area.
Keep the highest score, then suppress excessive overlap.
Repeat with the highest-scoring candidate still available.
NMS returns existing boxes without redrawing them.
Higher IoU thresholds suppress fewer candidates on identical input.
Class-wise NMS does not compare different predicted classes.
Score cutoffs and IoU thresholds filter different properties.
Greedy NMS does not optimize total retained score.
Crowds can cause real detections to be suppressed.