Slot Attention in Vision

Slots start as identical blank queries and compete for pixels through attention. After a few rounds each slot owns one object, with no boxes or labels needed.

Three slots settle at 0.71, 0.22, and 0.07 over the red square after three rounds

Why Does This Exist?

Unsupervised separation needs a mechanism that breaks symmetry: identical slots must end up owning different objects without any box telling them apart. Fixed spatial tiles cannot do it, because objects move across tiles.

Slot attention makes slots compete. Every region's features are claimed through normalized attention over slots (not over regions), so explaining a region well steals it from rival slots. A few recurrent rounds sharpen the split, and a shared decoder turns each slot into a mask. The parent map is object-centric learning; siblings are MONet and IODINE.

Think of It Like This

Three magnets under scattered pins

Shake pins over paper with three magnets underneath. Each pin snaps to the nearest strong magnet, and moving one magnet drags only its pins. Nobody labeled which pin belongs where; competition decided.

Slots are the magnets and pixel regions are the pins. Attention weights are the pull strengths, renormalized so every pin's total pull sums to 1 across magnets. The analogy stops at iteration: magnets pull once, while slots update their queries from what they won and pull again, sharper each round.

How It Actually Works

Let $N$ region features compete for $K$ slots. Queries $q_k$ from slots attend to keys from regions, and softmax runs over slots so each region's weights sum to 1: winners take regions from losers. Slots then update with a recurrent cell from their claimed regions, and the loop repeats 3 to 5 times. Each final slot decodes a mask plus pixels; masks mix by softmax into the reconstruction.

Worked example

A demo input holds 6 regions (red square over 2 regions, blue circle over 2, background 2) with $K = 3$ slots over 3 rounds. For a red-square region, round-1 weights read $[0.40, 0.35, 0.25]$ (undecided). By round 3 they settle at $[0.71, 0.22, 0.07]$ : slot 1 owns it at 0.71, the sum $0.71 + 0.22 + 0.07 = 1.00$ conserved. Final masks overlap the three objects at 0.89 mean. Dropping to 1 round leaves weights near $[0.45, 0.33, 0.22]$ and overlap at 0.61, so iteration carries the split.

Watch Out For

Reading slot order as object identity

Slots are an unordered set: slot 1 holds the square today and the circle tomorrow. The symptom is a tracker that follows slot index instead of slot content. Match slots across frames by mask overlap or features, never by index.

The Quick Version

Slots compete through attention normalized over slots, so regions are conserved.
Three to five recurrent rounds sharpen undecided weights into ownership.
On the demo input red-square weights settle from $[0.40, 0.35, 0.25]$ to $[0.71, 0.22, 0.07]$ .
Masks reach 0.89 overlap; one round stalls at 0.61.
Slots are unordered, so match by content across frames.