Weakly Supervised Learning

Weakly supervised learning draws boxes and masks from image-level tags using heatmaps and multi-instance rules instead of hand-drawn outlines.

A 64 by 64 heatmap thresholds at 0.5 to keep 1200 hot pixels whose bounding box localizes the tagged object

Why Does This Exist?

Drawing boxes around every dog in a million images costs months. Image-level tags saying dog or no dog cost almost nothing and already exist in search logs. The gap between cheap tags and expensive boxes blocks datasets that could be ten times larger.

Weakly supervised learning bridges it with localisation from classification. Class activation maps expose where the classifier looked, and multi-instance rules turn bag-level tags into pixel decisions. Programmatic labeling practice lives in weak supervision, which covers Snorkel-style rules rather than heatmaps. Detection craft lives in object detection.

Think of It Like This

Finding a hot pan by hovering hands

You cannot see which burner is on, but hovering hands feel heat rising strongest over one spot. You mark that hot zone and grab the handle beside it.

The classifier is the hovering hand, the heatmap is felt warmth, and the threshold picks the hot zone. The analogy stops at precision: hands find heat, not object edges, and heatmaps hug discriminative parts like faces over full bodies.

How It Actually Works

Train a classifier on image tags with global average pooling before the final layer, which preserves spatial maps. A class activation map weights each feature plane by its class weight and upsamples to image size. Thresholding at $0.5$ keeps hot pixels, connected components clean speckle, and the tight box around the largest component becomes the pseudo-box. Multi-instance learning frames it formally: an image is a bag, regions are instances, and a positive bag holds at least one positive instance.

Worked example

Heatmap $64 \times 64 = 4{,}096$ pixels for class dog. Threshold $0.5$ keeps $1{,}200$ hot pixels clustered over the torso and head. Largest component spans columns $18$ to $46$ and rows $20$ to $44$ , giving box $[18, 20, 46, 44]$ in heatmap cells, scaled by $8$ to $[144, 160, 368, 352]$ in a $512$ image. One tag became one box with no hand drawing.

Code

Threshold plus tight box from a heatmap.

1def hot_box(vals: list[float], width: int = 64, tau: float = 0.5) -> tuple[int, tuple[int, int, int, int]]:2    """Count hot cells and tight box over a flat heatmap."""3    hot = [i for i, v in enumerate(vals) if v >= tau]  # heat gate4    xs = [i % width for i in hot]5    ys = [i // width for i in hot]6    return len(hot), (min(xs), min(ys), max(xs), max(ys))7
8
9print(hot_box([0.9] * 1200 + [0.1] * 2896))  # -> (1200, (0, 0, 63, 18))

Watch Out For

Boxes that hug faces instead of bodies

Classifiers discriminate by parts: dog faces over torsos, bird heads over wings. The symptom is tight boxes around heads scoring well on classification but failing detection IoU.

Expand with erasing or multi-map fusion during training. Check box IoU on a small fully labeled probe, never classification accuracy alone.

The Quick Version

Weak supervision turns image tags into boxes and masks via heatmaps and bag rules.
CAM weights feature planes by class weights and upsamples to full resolution.
Worked numbers: $1{,}200$ of $4{,}096$ pixels kept at $0.5$ , box $[144, 160, 368, 352]$ .
Largest-component cleanup removes speckle before boxing.
Expect part-focus and verify against true boxes on a small probe set.