FCOS Detector

FCOS drops anchor boxes entirely and predicts each object's box straight from the feature point under its center, weighted by a centerness score.

FCOS predicts each object's box distances directly from the feature point at its center, with centerness down-weighting edge points.

Why Does This Exist?

Anchor boxes work but bring baggage: 9 templates per location, IoU matching thresholds, and dataset-specific clustering. FCOS, published in 2019, asked whether any of it is needed. Answer: treat every feature-map point inside a ground-truth box as a training sample and regress the four distances to the box edges directly. The result matched anchor-based accuracy with fewer hyperparameters, and kicked off the anchor-free wave.

Think of It Like This

Measuring a room from where you stand

Anchor methods hand you nine cardboard frames and ask which fits the room best. FCOS skips the frames: stand anywhere inside the room, pace off the distance to each of the four walls, and report the numbers.

Points near the room's middle pace evenly in all directions. Points near a wall give lopsided strides. Centerness is simply trusting the middle measurers more.

How It Actually Works

Per-pixel regression plus centerness

For a feature point at $(x, y)$ inside a ground-truth box, FCOS regresses $(l, t, r, b)$ , the distances to the left, top, right and bottom edges. Points outside every box train as background. A parallel centerness head predicts $\sqrt{\min(l,r)/\max(l,r) \times \min(t,b)/\max(t,b)}$ , which is 1 at the exact center and decays toward edges. Final scores multiply class confidence by centerness, so off-center predictions fade before NMS.

Multi-level assignment by size

Each pyramid level handles its own size band: small objects train on fine maps, large ones on coarse maps, with hard size cutoffs per level. This replaces anchor scale design with two numbers per level, far less tuning for the same small-object recall.

Code

1import torch2
3def centerness(l: torch.Tensor, t: torch.Tensor, r: torch.Tensor, b: torch.Tensor) -> torch.Tensor:4    return torch.sqrt((torch.min(l, r) / torch.max(l, r)) *5                      (torch.min(t, b) / torch.max(t, b)))6
7# A point 10px from left/top and 30px from right/bottom of its box:8print(round(centerness(torch.tensor(10.), torch.tensor(10.),9                       torch.tensor(30.), torch.tensor(30.)).item(), 3))  # -> 0.333

Watch Out For

Ambiguous points in overlapping boxes

Where two ground-truth boxes overlap, one feature point sits inside both and the targets conflict. FCOS assigns it to the smaller box by default. The symptom is missed large objects behind small foreground ones in crowds. Center-sampling, training only near each box's middle, shrinks the ambiguous zone.

Centerness mistaken for a second NMS

Centerness re-weights scores before NMS but does not replace it: duplicate boxes from neighboring points still survive without suppression. The symptom is clustered duplicates around each object. Keep an NMS or Soft-NMS step after the re-weighting.

The Quick Version

FCOS regresses four edge distances from every feature point inside an object.
Centerness scores favor middle points and fade edge predictions before NMS.
Pyramid levels split objects by size bands instead of anchor scales.
Overlapping boxes create ambiguous points, handled by smallest-box assignment plus center sampling.