IODINE for Scene Decomposition

IODINE refines all slots together over several inference steps. Each iteration compares the current render against the image and corrects every slot at once.

Four refinement rounds lift overlap 0.52 to 0.88 on overlapping shapes

Why Does This Exist?

One-pass encoders guess the split from a single glance, which fails where objects overlap: shared pixels get assigned once, and the hidden parts stay wrong. Real scenes need a second look that checks the current guess against the image.

IODINE (Iterative Object Decomposition Inference Network) treats slots as a posterior refined over steps. Each iteration renders the current slots, compares against the input, and updates all slot posteriors together. The parent is object-centric learning; unlike MONet it refines jointly rather than carving sequentially.

Think of It Like This

A sculptor circling the clay

A sculptor roughs out all figures at once, steps back, compares against the model, then corrects every figure a little. Four walk-arounds turn blobs into likenesses. Carving one figure perfectly first would crack the shared base.

IODINE circles the same way. Each iteration is a walk-around: render, compare, correct all slots. The analogy stops at memory: sculptors remember intent across rounds, while IODINE carries only the current posterior and gradients, with no symbolic plan of the scene.

How It Actually Works

Slots start from learned initial posteriors (means and variances per slot). Each of $T$ iterations samples slots, decodes each to a mask plus appearance, renders the mixture, and encodes the residual error back into posterior updates. Training maximizes a variational lower bound across steps, so early steps may stay blurry while late steps sharpen. Cost scales linearly with $T$ at inference.

Worked example

A demo scene holds 3 overlapping shapes with 5 slots over 4 iterations. Mean overlap climbs 0.52, 0.71, 0.82, then 0.88: gains of 0.19, 0.11, and 0.06, diminishing as expected. Two extra slots stay below 0.05 fill, absorbing background. Cutting to 1 iteration freezes overlap at 0.52 with merged masks on the overlap zone. Four steps cost four decoder passes for a 0.36 overlap gain; a fifth step adds only 0.02, so the demo stops at four.

Watch Out For

Paying for steps that stopped helping

Overlap plateaus while compute keeps billing per iteration. The symptom is 8-step configs matching 4-step quality at twice the latency. Sweep the step count on held-out scenes and ship the knee, not the maximum.

The Quick Version

IODINE refines all slot posteriors together over repeated render-and-compare steps.
Each iteration corrects every slot from the current residual error.
On the demo shapes overlap climbs 0.52 to 0.88 across four steps.
Extra slots absorb background below 0.05 fill instead of splitting objects.
Sweep step count and ship the knee where gains flatten.