Non Maximum Suppression
This concept covers the fundamentals of non maximum suppression within the broader context of Computer Vision.
Why Does This Exist?
When modern object detection models like YOLO or Faster R-CNN process an image, they do not cleanly output exactly one bounding box per object. Instead, they operate on a dense grid of anchors or proposals, meaning they often output dozens or even hundreds of bounding box predictions for the exact same object. If a person is walking down the street, the model might draw ten slightly different boxes around them, all with high confidence scores.
If we showed all these raw predictions to a user, the output would be a messy, illegible cluster of overlapping rectangles. Non-Maximum Suppression (NMS) exists to solve this problem. It acts as a post-processing filter that looks at all the predicted bounding boxes, identifies the groups of boxes that are pointing to the same object, and suppresses (discards) the redundant ones. It ensures that the final output is clean, with exactly one accurate bounding box for every real object detected in the image.
Think of It Like This
A group of people pointing out a star
Imagine you and ten friends are outside at night, and someone asks, "Where is the North Star?" All ten of you immediately point your fingers at the exact same star in the sky.
If a bystander were trying to figure out where the star is by looking at all eleven of your hands, the clustered overlapping fingers might be confusing. Instead, they look at who seems the most certain. The astronomer in your group is pointing very clearly and confidently. The bystander decides to trust the astronomer's finger as the "best" prediction and ignores the rest of your hands, knowing you are all pointing at the exact same thing anyway.
Non-Maximum Suppression does exactly this: when multiple bounding boxes "point" at the same object, it trusts the most confident box and ignores the others.
How It Actually Works
Non-Maximum Suppression is a greedy algorithmic loop. To run NMS, we need two pieces of information for every prediction: the bounding box coordinates and the confidence score (the model's certainty that the box contains an object).
1. Sort by Confidence
First, we take all the predicted bounding boxes for a specific class (e.g., all boxes predicting "car") and sort them in descending order based on their confidence scores. The box with the highest confidence score goes to the top of the list.
2. Pick the Maximum
We select the box at the very top of the list (the "maximum"). We remove it from our pending list and add it to our final list of approved detections. Because it has the highest confidence, we assume it is the most accurate bounding box for that particular object.
3. Calculate Overlap (IoU)
Next, we compare our chosen "maximum" box against all the remaining boxes in the pending list. We measure how much they overlap using a metric called Intersection over Union (IoU). IoU calculates the area of overlap between two boxes divided by the total area they both cover. The result is a number between 0 (no overlap) and 1 (perfect overlap).
4. Suppress Redundant Boxes
We check the IoU of each remaining box against a predefined threshold (commonly 0.4 or 0.5). If a remaining box overlaps with our chosen box by more than the threshold, we assume it is detecting the exact same object. We permanently discard (suppress) this overlapping box.
5. Repeat
We return to step 2. We find the box with the highest confidence among the ones that survived the suppression, add it to our final detections, and suppress any remaining boxes that heavily overlap with it. We repeat this loop until the pending list is entirely empty.
Code
def non_maximum_suppression(boxes: list, scores: list, iou_threshold: float) -> list: # boxes: list of [x1, y1, x2, y2], scores: confidence probabilities # -> Returns list of indices to keep # Start with a list of indices sorted by confidence score (highest first) pending = sorted(range(len(scores)), key=lambda i: scores[i], reverse=True) keep = [] while pending: # 1. Pick the box with the highest score current = pending.pop(0) keep.append(current) # 2. Compare with the rest and keep only those with low IoU survivors = [] for other in pending: iou = calculate_iou(boxes[current], boxes[other]) # 3. Suppress if the overlap is too high if iou <= iou_threshold: survivors.append(other) pending = survivors return keepWatch Out For
Setting the IoU Threshold Too High or Too Low
The IoU threshold is a crucial hyperparameter in NMS.
If you set it too high (e.g., 0.9), NMS will only suppress boxes that are nearly identical. If the model outputs two boxes that overlap heavily but not perfectly, both will survive, leaving you with duplicate detections for the same object.
If you set it too low (e.g., 0.1), NMS becomes too aggressive. If two different people are standing very close to each other in a photo, NMS might mistake their bounding boxes for duplicates and suppress one of them, causing the model to completely miss a real person.
The Quick Version
- Object detection models naturally generate multiple overlapping bounding boxes for a single object.
- Non-Maximum Suppression (NMS) cleans up this output by keeping only the best box and discarding the rest.
- It selects the bounding box with the highest confidence score and adds it to the final output.
- It then calculates the Intersection over Union (IoU) between that top box and all other remaining boxes.
- Any box that overlaps with the top box beyond a certain threshold is considered a duplicate and is permanently suppressed.