Soft-NMS Filtering

Soft-NMS decays the scores of overlapping boxes with a smooth penalty instead of deleting them, so crowded objects survive with lower confidence rather than vanishing.

Soft-NMS lowers the scores of overlapping boxes with a smooth penalty, keeping crowded detections alive at reduced confidence.

Why Does This Exist?

Hard non-maximum suppression deletes every box overlapping the top pick above threshold. In crowds, two real pedestrians overlap heavily, and the second person gets erased: a miss no classifier can appeal. Soft-NMS, published in 2017, replaces deletion with decay. Overlapping boxes lose score smoothly instead of dying, so the runner-up survives at reduced confidence and the final threshold decides. Same loop, one-line change, consistent 1-point AP gains on COCO with bigger wins in crowds.

Think of It Like This

A demotion instead of a firing

Hard NMS fires anyone standing too close to the star employee. Soft-NMS demotes them: stand close to the star and your review score drops, but a genuinely strong second performer still clears the bar.

Crowded scenes are full of strong second performers. Demotion keeps them employed at lower rank instead of losing them entirely.

How It Actually Works

Gaussian penalty in the same loop

Sort by score, pick the max, then rescore instead of suppress. With the Gaussian form, each remaining box score $s_i$ becomes $s_i \times e^{-\text{IoU}^2 / \sigma}$ , where $\sigma$ is typically 0.5. A box at IoU 0.8 with the top pick keeps $e^{-0.64} \approx 0.53$ of its score; at IoU 0.3 it keeps $e^{-0.09} \approx 0.91$ . Distant boxes barely notice, near-duplicates fade hard, and true neighbors persist.

A worked comparison

Kept box K scores 0.9. Neighbor M scores 0.75 at IoU 0.816 with K; distant box N scores 0.8 at IoU 0. Hard NMS at threshold 0.5 deletes M outright. Soft-NMS with $\sigma = 0.5$ rescales M to $0.75 \times e^{-0.816^2/0.5} = 0.75 \times e^{-1.332} \approx 0.75 \times 0.264 \approx 0.198$ , while N keeps its full 0.8. M survives for a low threshold to catch and vanishes under a strict one, which is exactly the desired softness.

Code

1import math2
3def soft_nms_decay(score: float, iou: float, sigma: float = 0.5) -> float:4    return score * math.exp(-(iou ** 2) / sigma)5
6print(round(soft_nms_decay(0.75, 0.816), 3))  # -> 0.1987print(round(soft_nms_decay(0.80, 0.0), 3))    # -> 0.8

Watch Out For

Sigma copied blindly into dense scenes

Sigma 0.5 fits COCO crowds; in ultra-dense crowds it under-decays and duplicates survive, while in sparse scenes it pointlessly softens clean picks. The symptom is duplicate boxes or no change versus hard NMS. Sweep sigma on your own density before shipping.

Extra boxes slowing the pipeline

Soft-NMS keeps boxes hard NMS would delete, so downstream tracking or rendering sees more candidates. The symptom is a slower tracker after a painless detector swap. Re-tune the final score threshold to absorb the survivors.

The Quick Version

Soft-NMS decays overlapping box scores instead of deleting the boxes.
Gaussian penalty $e^{-\text{IoU}^2/\sigma}$ fades near-duplicates and spares distant boxes.
Crowded objects survive at reduced confidence instead of vanishing.
Sigma and the final threshold both need tuning per scene density.