Skip to content
AI360XpertBeta
Computer Vision

Visual explainer

U-Net Segmentation

Follow spatial scales, skip connections, channel mixing, and pixel predictions.

1. Predict At Every Pixel

Segmentation predicts a class at each pixel rather than one label for the image.
Segmentation predicts a class at each pixel rather than one label for the image.

U-Net is an encoder-decoder network for image segmentation. Instead of answering only whether a cell is present, it predicts which pixels belong to the cell. The grid is a small teaching example, not a measured model result.

2. Gather Context By Downsampling

Four downsampling steps reduce a 256-square feature map to 16-square.
Four downsampling steps reduce a 256-square feature map to 16-square.

Follow all four halvings: 256, 128, 64, 32, 16. Convolutions learn features between pooling steps. These dimensions describe a same-padded example; the original architecture also shrinks feature maps through unpadded convolutions.

3. Restore Spatial Resolution

The decoder restores spatial resolution from 16-square to 256-square.
The decoder restores spatial resolution from 16-square to 256-square.

The expanding path upsamples through the matching scales. More output locations do not automatically mean more recovered information: the decoder must learn how to use coarse context and features arriving through the skip connections.

4. Connect Matching Scales

Skip connections carry encoder features to decoder stages at matching spatial scales.
Skip connections carry encoder features to decoder stages at matching spatial scales.

Trace a horizontal bridge alongside the longer route through the bottleneck. Encoder features provide spatial information that can help localization. They are learned feature maps, not ready-made outlines, and their usefulness depends on training.

5. Concatenate Along Channels

Concatenating 64 encoder channels with 64 decoder channels produces 128 channels.
Concatenating 64 encoder channels with 64 decoder channels produces 128 channels.

At this illustrative stage, both inputs share height and width. Concatenation stacks their channels; it does not add corresponding values. The decoder now receives both sets of features for subsequent convolutions to combine.

6. Learn How To Combine Features

Learned convolutions mix copied encoder features with decoder features.
Learned convolutions mix copied encoder features with decoder features.

The tiny calculation isolates channel mixing: 0.5 × 2 − 0.25 × 1 = 0.75. A spatial convolution also combines neighbouring positions. Training learns these weights; copying features across a bridge alone does not produce a segmentation.

7. Produce A Pixel Mask

A binary prediction head turns per-pixel probabilities into a foreground mask.
A binary prediction head turns per-pixel probabilities into a foreground mask.

For a binary variant, a 1 × 1 output convolution can produce logits, followed by sigmoid. The example applies a 0.5 threshold. Multiclass segmentation instead predicts class scores at each pixel, commonly using softmax.

8. Account For The Borders

Padding determines whether a convolution preserves spatial size.
Padding determines whether a convolution preserves spatial size.

For one stride-one convolution, output size is input size + 2 × padding − kernel size + 1. The original U-Net paper uses valid convolutions and crops encoder features before concatenation; same-padded variants handle dimensions differently.

9. Understand What Pooling Loses

Max pooling keeps the strongest activation but loses its exact position.
Max pooling keeps the strongest activation but loses its exact position.

Both patches produce the same maximum. From that value alone, the decoder cannot know which corner was active. Skip features offer another route for spatial information, but fine structures and boundaries still require suitable data, resolution, and training.

10. Separate Classes From Identities

Semantic class labels do not encode individual object identities.
Semantic class labels do not encode individual object identities.

A foreground label says “cell,” not “cell A.” Semantic predictions can learn gaps between touching objects, as the original work demonstrates. Assigning persistent object identities requires an instance representation or additional processing.

The Quick Version

  • Segmentation assigns a class to each pixel.
  • Four halvings take 256 through 32 to 16.
  • Upsampling restores resolution, not automatically lost information.
  • Skip connections supply encoder features at matching scales.
  • Concatenation appends channels; it does not add values.
  • Learned convolutions combine spatial and channel information.
  • A prediction head produces pixel scores and labels.
  • Padding determines output size and skip alignment.
  • Pooling can discard precise locations.
  • Semantic classes and instance identities answer different questions.