FixMatch Semi-Supervised

FixMatch turns confident guesses on weakly augmented images into training labels for their strongly augmented twins.

Weak views guess 41 confident labels above 0.95 from 128 images and strong views train against them beside 32 true labels

Why Does This Exist?

Labeling tens of thousands of images is slow, but unlabeled images are free. Plain pseudo-labeling feeds the model its own mistakes: early errors become training truth and the loop locks in. Teams need a filter that keeps only guesses worth teaching.

FixMatch pairs two views of each unlabeled image. A weakly augmented crop guesses the label, and only guesses above high confidence teach the strongly augmented twin. The umbrella is semi-supervised learning, the low-data cousin is prototypical networks, and the task is usually image classification.

Think of It Like This

A fair copy from a clean draft

A student writes a clean draft first, circles only the answers they feel sure about, then copies those answers onto a messy worksheet with coffee stains. The stains must not change the answers.

Weak views are the clean draft, confidence is the circling, and strong augmentation is the stains. The analogy stops at intent: students want practice, while FixMatch wants the model invariant to distortion.

How It Actually Works

Each batch mixes a few labeled images with many unlabeled ones. Labeled images train with plain cross-entropy. Each unlabeled image gets a weak view (flip and crop) whose top probability must clear threshold $\tau = 0.95$ to become a pseudo-label, plus a strong view (heavy colour and geometric distortion) trained against that label. Unconfident images contribute nothing that step, which is the whole defence against error feedback.

Worked example

Batch: $32$ labeled plus $128$ unlabeled. Weak views score $41$ images above $0.95$ , about one third. Suppose supervised loss is $0.80$ and unsupervised loss over the $41$ kept is $0.60$ with weight $1.0$ : total $1.40$ . The other $87$ images cost nothing this step but may clear the bar next epoch as the model sharpens.

Code

Mask and loss split for one batch of max probabilities.

1def fixmatch_mask(max_probs: list[float], tau: float = 0.95) -> tuple[int, float]:2    """Count kept images and kept share; unconfident ones teach nothing."""3    kept = sum(1 for p in max_probs if p >= tau)  # hard gate, no soft weight4    return kept, kept / len(max_probs)5
6
7print(fixmatch_mask([0.97] * 41 + [0.6] * 87))  # -> (41, 0.32)

Watch Out For

Lowering the threshold to use more data

Dropping $\tau$ to $0.8$ doubles kept images and fills them with errors. The symptom is fast early gains then a plateau below the supervised baseline as mistakes fossilise.

Keep $0.95$ until the end. Warm up on labeled data first so early pseudo-labels start clean.

The Quick Version

FixMatch teaches strongly augmented views from confident weak-view guesses.
Threshold $\tau = 0.95$ gates every pseudo-label; the rest teach nothing that step.
Worked numbers: $41$ of $128$ kept, about one third, total loss $1.40$ .
Labeled and unlabeled losses simply add with equal weight.
Keep the threshold high and warm up supervised first.