Anchor-Free Detection

Anchor-free detection predicts boxes from points or keypoints instead of preset templates, removing thousands of hand-tuned anchors per image.

Anchor-free detectors predict boxes from feature points and keypoints directly, replacing thousands of preset anchor templates.

Why Does This Exist?

Anchor boxes tile every image with thousands of preset templates, each needing scale, aspect ratio, IoU thresholds and dataset clustering. That machinery is brittle on new domains and pure overhead on deployment. Anchor-free detection, rising from CornerNet in 2018 through FCOS and CenterNet in 2019, predicts geometry straight from feature locations: corners, centers or per-pixel distances. Fewer knobs, same accuracy, faster adaptation to odd datasets like aerial or medical imagery.

Think of It Like This

Sketching freehand versus tracing stencils

Anchor methods trace: pick the closest of nine stencils per spot, then erase and adjust. Anchor-free methods sketch freehand: mark the corners or the middle, then draw the sides.

Stencils speed up beginners on familiar shapes and slow down everyone on strange ones. Freehand demands better drawing but handles any shape it meets.

How It Actually Works

The three anchor-free shapes

Corner pairs. CornerNet predicts top-left and bottom-right heatmaps plus embeddings that pair corners of the same object. Accurate corners give tight boxes, but pairing fails in crowds.

Center plus size. CenterNet predicts one center heatmap and regresses width and height per peak. Simplest to deploy, weakest on overlapping centers.

Per-pixel distances. FCOS regresses four edge distances from every interior feature point, filtered by centerness. Densest supervision signal of the three, with ambiguity where boxes overlap.

What disappears, what stays

Gone: anchor clustering, IoU matching thresholds, per-level scale tables. Staying: multi-scale pyramids, classification losses, and some form of duplicate suppression. Anchor-free removes the template bureaucracy, not the detection fundamentals, which is why modern hybrids freely mix both ideas.

Code

1# Anchor-free decoding: edge distances from one interior point become a box2import torch3
4point = torch.tensor([160., 140.])          # feature point inside the object5ltrb = torch.tensor([60., 40., 80., 60.])   # left, top, right, bottom distances6
7x1, y1 = point[0] - ltrb[0], point[1] - ltrb[1]8x2, y2 = point[0] + ltrb[2], point[1] + ltrb[3]9box = torch.stack([x1, y1, x2, y2])         # -> [100, 100, 240, 200]

Watch Out For

Declaring anchors dead on every dataset

On datasets with extreme aspect ratios, like long thin defects, tuned anchors still beat point methods because the template prior carries real information. The symptom is ragged boxes on skinny objects after migrating to anchor-free. Let the data decide: benchmark both on your own boxes.

Forgetting duplicates still need suppression

Anchor-free removes templates, not duplicate predictions: neighboring points still fire on the same object. The symptom is clustered boxes around each detection. Keep NMS, Soft-NMS or peak extraction in the pipeline.

The Quick Version

Anchor-free detection predicts boxes from corners, centers or per-pixel distances.
It removes anchor scales, aspect ratios and IoU matching thresholds.
Pyramids, classification losses and duplicate suppression all stay.
Tuned anchors can still win on extreme aspect ratios, so benchmark per dataset.