DreamBooth

DreamBooth teaches a diffusion model your specific subject from three to five photos by binding it to a rare word while rehearsing the general class.

Why Does This Exist?

Text-to-image models know dogs in general and your dog not at all. Prompting harder cannot summon Buster's crooked ear because the model never saw it. Full fine-tuning on four photos instead destroys the model's broader knowledge within minutes, a textbook overfit: Buster everywhere, everything else forgotten.

DreamBooth threads the needle with two losses. A reconstruction loss binds the subject to a rare identifier like [V] dog, and a prior-preservation loss keeps generating ordinary dogs from the plain word dog. The tiny-adapter alternative that freezes the base model is covered by the LoRA for image models owner page, and the embedding-only cousin is textual inversion.

Think of It Like This

Teaching a name without erasing the language

You teach a child that your neighbour's poodle is called Biscuit. You show four photos of Biscuit, but between lessons you keep reading ordinary dog books together so the word dog still means all dogs. Months later the child draws Biscuit on the moon and also still draws generic dogs on demand.

The rare token is the name Biscuit. The class rehearsal is the continued dog-book reading. Skip the books and the child concludes every dog is Biscuit, which is exactly the language drift the prior loss prevents.

How It Actually Works

Pick a rare token [V] with weak prior meaning and prompt training pairs as a photo of a [V] dog for $3$ – $5$ subject photos. Simultaneously generate or collect around $200$ class images of ordinary dogs captioned a photo of a dog. Fine-tune the UNet, and usually the text encoder, on both streams:

L = L_{\text{subject}}([V]) + \lambda \, L_{\text{prior}}(\text{class}), \qquad \lambda \approx 1

The subject loss teaches Buster's identity; the prior loss rehearses generic dogs so the update cannot collapse the class onto one instance. Training runs about a thousand steps, minutes on a single GPU, because the dataset is tiny by design.

Worked example

Four Buster photos plus $200$ generated class dogs, $\lambda = 1.0$ . Each step samples one stream or the other, so about two percent of gradient comes from Buster and ninety-eight from generic dogs, yet Buster's four photos repeat hundreds of times across training while class images cycle. After training, a [V] dog on the beach renders Buster's markings in a new scene, while a dog on the beach still renders varied strangers. Drop the prior stream and within $200$ steps every dog prompt returns Buster.

Code

Batch composer showing the subject-to-class sampling ratio.

1def expected_subject_steps(steps: int = 1000, n_subject: int = 4, n_class: int = 200) -> float:2    """Mean steps touching the subject under proportional sampling."""3    return steps * n_subject / (n_subject + n_class)4
5
6print(round(expected_subject_steps(), 1))  # -> 19.6 of 1000 steps see Buster

Watch Out For

Skipping prior preservation to save minutes

Generating $200$ class images takes one short run, and skipping it feels efficient. Without it the model overfits fast: backgrounds from the four photos glue onto the subject, and the class word drifts until every dog is Buster.

Always generate the class set from the frozen model before fine-tuning. The few minutes spent pay back in every prompt that is not about your subject.

Choosing a common word as the identifier

Binding to dog directly or a meaningful word like special tangles the subject with existing associations, and prompts leak subject features into unrelated generations. Rare tokens exist precisely to offer empty meaning.

Use a rare token string and verify it has weak priors by generating from it before training: outputs should be generic and varied.

The Quick Version

DreamBooth fine-tunes on $3$ – $5$ subject photos captioned with a rare token like [V] dog.
A prior-preservation loss on around $200$ class images stops the class collapsing onto the subject.
Our schedule shows about $20$ of $1000$ steps touching the subject, enough with repetition.
Rare tokens prevent leaking subject traits into ordinary words.
Skipping class rehearsal overfits within $200$ steps; common-word identifiers tangle meanings.