Classical Image Segmentation

Classical segmentation splits foreground from background with pixel cutoffs, edge maps, and grown regions, no neural network involved. It works wherever labelled training data is scarce and every decision must stay explainable.

One six-pixel strip takes three classical routes to the same split: a cutoff at 100, a gradient boundary, and growth from two seeds

Why Does This Exist?

Before neural networks, separating an object from its background had to work with arithmetic alone: no learned features, no training set, just pixel values and their layout. Those methods still earn their keep in medical imaging and industrial inspection, where annotated data is scarce and each decision must be explainable to an auditor. This page maps the three classical routes and when each wins. The cutoff route gets its full derivation in Otsu thresholding, separating touching objects via flooding in watershed segmentation, the interactive extension in GrabCut, and the learned successor that replaced all three in encoder-decoder segmentation architectures.

The one idea you need first

A grayscale image is a grid of intensities, typically $0$ for black to $255$ for white. Foreground and background usually sit in different intensity bands, so a well-chosen cutoff already separates most pixels correctly. Everything below is a strategy for choosing that cutoff, finding the boundary directly, or growing it from a known point. Colour versions of the same idea start by picking a space where the bands pull apart, covered in colour spaces in vision.

Think of It Like This

Sorting laundry by shade, then by seams, then by pile

Dump a mixed laundry basket on the bed. First pass: whites in one pile, darks in another, judging each garment alone. That is thresholding. Second pass: run a finger along every seam and pocket edge to find where one garment ends and the next begins. That is edge detection. Third pass: pick one certain white sock, then keep adding touching garments that feel equally white until the pile stops growing. That is region growing.

The analogy stops at touching lookalikes: a white shirt lying on white sheets joins the wrong pile by every one of these rules, because none of them knows what a shirt is.

How It Actually Works

Three routes take the same pixels to a foreground mask, and they fail in different places, which is why toolkits keep all three.

Thresholding: one cutoff for every pixel

Pick a threshold $T$ . Pixel $(x, y)$ with intensity $I(x, y)$ becomes foreground ( $1$ ) when $I(x, y) > T$ , background ( $0$ ) otherwise. The hard part is choosing $T$ : Otsu's method tries every candidate and keeps the one that minimizes within-class variance, the weighted spread of the two groups it creates. Tight groups mean a clean split.

Edge detection: find the boundary, fill the inside

Objects end where brightness jumps, so compute the gradient magnitude $G = \sqrt{G_x^2 + G_y^2}$ per pixel with an operator like Sobel or the Canny pipeline, then close the gaps with morphological operations to get sealed contours. Sealed contours fill into masks. This route finds boundaries the cutoff route misses, but one gap in the contour floods the fill.

Region growing: expand from a known seed

Plant a seed pixel known to be foreground, then repeatedly absorb neighbours whose intensity sits within a tolerance of the region's running mean. Growth halts where the tolerance fails, which traces the object's outline without ever scanning the whole image. The seed is the method's strength and its trap: a seed on the wrong texture grows the wrong object confidently.

Worked example

Six pixels in a strip read $[20, 25, 200, 210, 205, 30]$ . At cutoff $T = 100$ the mask is $[0, 0, 1, 1, 1, 0]$ . Background $\{20, 25, 30\}$ has mean $25.0$ and foreground $\{200, 210, 205\}$ has mean $205.0$ , each with variance $16.67$ , so the within-class variance is $16.67$ . A worse split that strands $30$ with the bright group scores $3838.54$ on the same measure, which is why Otsu settles on $100$ . The edge route sees the same answer as the two jumps, $25 \to 200$ and $205 \to 30$ . The region route plants a seed on the $200$ and absorbs $210$ and $205$ before the tolerance rejects $25$ and $30$ .

Code

Threshold plus the Otsu score for the winning split, in plain NumPy:

1import numpy as np2
3px = np.array([20, 25, 200, 210, 205, 30])4mask = (px > 100).astype(int)5print(mask.tolist())  # -> [0, 0, 1, 1, 1, 0]6
7bg, fg = px[mask == 0], px[mask == 1]8within = (bg.var() * bg.size + fg.var() * fg.size) / px.size9print(round(float(bg.mean()), 2), round(float(fg.mean()), 2))  # -> 25.0 205.010print(round(float(within), 2))  # -> 16.67

Watch Out For

One global cutoff under uneven light

A single $T$ assumes the lighting is flat, so a shadow across half the image sorts dark foreground with bright background indiscriminately. The symptom is a mask that follows the shadow's edge instead of the object's. Estimate the cutoff in local windows that ride the illumination gradient instead of one global value.

Expecting the segment to know the object

Classical routes group by visual similarity, never by meaning. A dog on a same-textured rug merges into one region, and no tolerance tweak fixes it because the pixels genuinely match. When the task needs object identity rather than uniform patches, switch to a learned segmenter instead of tuning cutoffs.

The Quick Version

Thresholding labels each pixel by one intensity cutoff, and Otsu picks the cutoff that minimizes within-class variance.
Edge routes find boundaries from gradient jumps, then seal and fill them into masks.
Region growing absorbs similar neighbours outward from a seed until a tolerance stops it.
Six pixels at cutoff 100 split into groups with variance 16.67 against 3838.54 for a bad split.
Uneven lighting breaks global cutoffs, and none of the three routes understands object identity.