DETR Detects Sets Without Anchors

DETR predicts a fixed set of boxes with transformers and matches them one to one, removing anchors and NMS entirely.

One hundred queries match two truths at cost minus 1.54 with no anchors

Why Does This Exist?

Anchors need scales, ratios, IoU thresholds, and NMS tuning per dataset. Crowded scenes break the heuristics first.

DETR drops them. It builds on object detection set framing, uses attention mechanisms globally, and leads to deformable decoders.

Think of It Like This

Assigned parking instead of first come lots

Open lots need circling plus towing of doubles. Assigned bays give each car one spot with no towing.

Queries are bays, Hungarian is the attendant. Where it stops: too few bays overflow, just as too few queries miss dense crowds.

How It Actually Works

Backbone makes $d = 256$ maps. Encoder adds sine positions. Decoder starts $N = 100$ learned queries, attends to memory, and outputs class plus box $\hat{b} = (cx,cy,w,h)$ . Match cost is $-p + L1 + GIoU$ , solved by Hungarian in $O(N^3)$ .

Worked example

$N = 3$ , truths cat $[0.2,0.2,0.4,0.4]$ and dog $[0.7,0.7,0.3,0.3]$ . Predictions give costs $[[-0.79,1.83,0.50],[1.83,-0.75,2.20],[0,0,0]]$ where $-0.79 = -0.85 + 0.06$ and $-0.75 = -0.80 + 0.05$ . Best picks $(1,1),(2,2)$ total $-1.54$ . No duplicates remain, so NMS has nothing to do.

Watch Out For

Training 50 epochs and calling DETR broken

DETR needs about 500 epochs to settle matching. Symptom: low AP at 50 epochs blamed on code. Fix: train long or move to deformable variants before judging.

The Quick Version

DETR predicts $N = 100$ boxes as a set with no anchors.
Hungarian matching pairs each truth once at $O(N^3)$ .
Fixture cost minus 1.54 picks cat and dog correctly.
Removal of NMS helps crowded scenes most.
Budget 500 epochs or use faster variants.