DCGAN

DCGAN showed that a generator built only from strided convolutions and batch normalisation can learn to draw convincing bedroom photos from random noise.

DCGAN grows a 100 number noise vector into a 64 pixel photo through four doublings of strided convolution

Why Does This Exist?

The original GAN proved the game could work but gave almost no architectural guidance. Generators built from fully connected layers produced blurry digits and collapsed constantly, and nobody knew which choices mattered. Scaling to real photos looked hopeless.

DCGAN replaced folklore with a recipe. All-convolutional stacks, strided convolutions instead of pooling, batch normalisation almost everywhere, ReLU in the generator with Tanh at the output, LeakyReLU in the discriminator. On bedroom photos those rules produced the first GAN samples that looked like places rather than textures. Every convolutional GAN after it, including conditional GAN and the paired translator Pix2Pix, starts from this template.

Think of It Like This

Developing a photo from a thumbnail sketch

A darkroom printer starts from a tiny blurry thumbnail and enlarges it in stages. Each enlargement doubles the size while adding finer grain, and a chemical bath between stages keeps contrast stable so no stage blows out to white or black.

The noise vector is the thumbnail. Each strided transposed convolution doubles the resolution, and batch normalisation is the stabilising bath. Skip the bath and contrast explodes by the fourth doubling, which is exactly the training collapse the recipe prevents.

How It Actually Works

The generator takes $z$ , a $100$ -dimensional vector of uniform noise, projects it to $4 \times 4 \times 1024$ , then applies four fractionally-strided convolutions that double spatial size while quartering depth: $4 \times 4 \times 1024 \to 8 \times 8 \times 512 \to 16 \times 16 \times 256 \to 32 \times 32 \times 128 \to 64 \times 64 \times 3$ . The discriminator mirrors this with strided convolutions down to a single real-or-fake score. No pooling, no fully connected hidden layers anywhere.

The activation rules carry real weight. Batch normalisation on every layer except the generator output and discriminator input keeps activations centred as the game shifts underneath both networks. ReLU lets the generator build unbounded bright features, Tanh bounds the final photo to $[-1, 1]$ , and LeakyReLU with slope $0.2$ keeps discriminator gradients alive for negative inputs where plain ReLU would go silent.

Worked example

First generator layer: project $100$ noise numbers to $4 \times 4 \times 1024 = 16{,}384$ values. That matrix holds $100 \times 16{,}384 = 1{,}638{,}400$ parameters, more than the rest of the generator combined, which is why later work shrank this projection first. Each doubling stage then roughly quarters parameters while quadrupling pixels, so capacity concentrates exactly where the image is smallest and most abstract.

Code

Shape trace of the generator, the quick check that catches a miswired doubling stage.

1def dcgan_shapes(z_dim: int = 100) -> list[tuple[int, int, int]]:2    """Spatial shapes through the four doubling stages."""3    shapes = [(4, 4, 1024)]4    h, w, c = shapes[0]5    for _ in range(4):  # each stage doubles size, quarters depth6        h, w, c = h * 2, w * 2, max(c // 4, 3)7        shapes.append((h, w, c))8    return shapes9
10
11print(dcgan_shapes())  # -> [(4, 4, 1024), (8, 8, 256), ..., (64, 64, 3)]12print(100 * 4 * 4 * 1024)  # -> 1638400 first-layer parameters

Watch Out For

Removing batch normalisation to save memory

The recipe's most load-bearing rule is batchnorm everywhere except the two boundary layers. Drop it and the generator's activations drift as the discriminator improves, gradients explode by the third doubling, and samples collapse to a single texture within epochs.

If memory forces a change, swap in instance or layer normalisation, never nothing. Unnormalised DCGAN at $64$ pixels is the canonical collapse demo.

Feeding photos in [0, 1] to a Tanh output

The generator output passes through Tanh, so it lives in $[-1, 1]$ . Normalise training photos to $[0, 1]$ and the discriminator separates real from fake by range alone in the first epoch, then never learns anything about content.

Scale real images to $[-1, 1]$ to match. This one-line mismatch has wasted more GAN training runs than any architecture choice.

The Quick Version

DCGAN builds both networks from strided convolutions only, no pooling or dense hidden layers.
A $100$ -dimensional noise vector grows through four doublings into a $64 \times 64 \times 3$ photo.
Batch normalisation on all but two layers plus ReLU, LeakyReLU, and Tanh makes training stable.
The first projection matrix alone holds $1{,}638{,}400$ parameters.
Match real images to the $[-1, 1]$ Tanh range or the discriminator cheats on range alone.