YOLO-World Fast Open Set

Add text embeddings to YOLO necks so one fast model answers new phrases live. Speed stays near classic YOLO.

YOLO-World adds text paths to a fast YOLO detector

Why Does This Exist?

Open detectors run slow transformer stacks, missing robotics needs. YOLO-World keeps the YOLO architecture grid plus a text neck, then folds text into conv weights for fixed vocab runs. Live phrase detection follows.

This page covers fast text fusion only. Heavy open detectors are sibling pages.

The one idea you need first.

Text neck mixes phrase vectors $t$ into multi-scale maps $v_s$ . Reparameterization folds fixed phrases into conv kernels, so runtime matches YOLO. New phrases rerun the fold cheaply.

Think of It Like This

A short-order grill with tickets on the rail

A grill cooks set meals fast. New tickets clip to the rail and cooks season on the fly without slowing the line. Regular orders pre-mix spices into stations.

YOLO grid is the grill, phrases are tickets, and folding is pre-mixing. Where it stops: kitchens remake plates, while detectors score all cells at once.

How It Actually Works

Encode phrases to $t$ . Fuse into PAN neck maps with max-sigmoid attention. Train contrastive box-phrase plus box regression on Objects365 plus GoldG. For fixed lists, fold $t$ into head convs; for new lists, recompute the fold in seconds.

In plain words: train open, fold for speed, and refold for new words.

Worked example

Large model LVIS rare AP $22.1$ at $52$ FPS on V100 versus $20.2$ at $9$ FPS for a heavy rival. The $43$ FPS gap with $1.9$ AP lead is the speed-open trade won. Small variant keeps $35.4$ AP at $74$ FPS.

Watch Out For

Rare phrases missing from fold lists

Folded heads answer only listed phrases and silently miss new ones. Live tests with fresh words flop. Keep the open path live or refold lists before evals in open-vocabulary detection.

The Quick Version

YOLO-World adds text necks to fast YOLO grids.
Grounding mixes train open phrases.
Folding text into convs keeps 52 FPS inference.
New phrases need a quick refold.
Speed plus openness suits live use.