Skip to content
AI360XpertBeta

Few-Shot Learning

Classify a new category correctly after seeing only a handful of labeled examples of it. A strong embedding does the hard work, so three examples can be enough.

Two examples per class average to prototypes at 1,0 and 1,4, and a query at 1,1 takes the nearer label by squared distances 1 versus 9
Two examples per class average to prototypes at 1,0 and 1,4, and a query at 1,1 takes the nearer label by squared distances 1 versus 9

Why Does This Exist?

A factory spots a new surface defect on Monday. The inspector has three phone photos. A normal classifier wants hundreds of examples per class, which means waiting months while bad parts ship.

Few-shot learning covers that gap. It classifies a genuinely new category from a handful of labeled examples, commonly one to ten, by leaning on an embedding learned earlier from many other categories.

One idea underneath it. Transfer learning reuses a representation trained elsewhere instead of starting from zero. Contrastive learning is one way to build that representation so similar things sit close together. This page covers the prototype rule used once that space exists. The prompt only version for large models lives in in-context learning.

Think of It Like This

Sorting new shells with three samples per bucket

You volunteer at a beach cleanup with two buckets for two new shell types. A marine biologist hands you three samples of each and points to two empty spots on the table as the home for each type.

New shells arrive all afternoon. You hold each one over both home spots and drop it where it looks closer. You never relearn what a shell is. You only compare against the two home spots built from the first three samples each.

Each sample shell is one support example, each home spot is one prototype average, holding up a new shell is embedding a query, and closer means smaller distance in that space.

Where it stops: you can turn a shell and check its back. A prototype classifier sees only one fixed vector per item and cannot ask for a better view.

How It Actually Works

Call the support set the few labeled examples per class. Call the query one new unlabeled example from the same classes. Write nn for shots per class and kk for ways, meaning class count, so two shots of two classes is 22-way 22-shot. Call ff the embedding function that maps any example to a vector. Call ckc_k the prototype of class kk.

Average each class support into its prototype, then label the query by the nearest prototype:

ck=1n∑i=1nf(xi),y^=arg⁡min⁡k∥f(xquery)−ck∥2c_k = \frac{1}{n} \sum_{i=1}^{n} f(x_i), \quad \hat{y} = \arg\min_k \lVert f(x_{\text{query}}) - c_k \rVert^2

Here ∥⋅∥2\lVert \cdot \rVert^2 is squared Euclidean distance, meaning sum of squared gaps per dimension. All the skill sits in ff. When ff already puts same class items close, averaging steadies the center and a nearest lookup is enough. When ff is poor, no prototype trick saves it, which is why teams borrow ff from transfer learning or contrastive learning and judge distance methods like k-nearest neighbors on the same space.

Worked example: two ways, two shots, plain points

Class A support points are [0,0][0, 0] and [2,0][2, 0]. Average them:

cA=[(0+2)/2,(0+0)/2]=[1,0]c_A = [(0 + 2) / 2, (0 + 0) / 2] = [1, 0]

Class B support points are [0,4][0, 4] and [2,4][2, 4]. Average them:

cB=[(0+2)/2,(4+4)/2]=[1,4]c_B = [(0 + 2) / 2, (4 + 4) / 2] = [1, 4]

Query q=[1,1]q = [1, 1]. Squared distance to cAc_A is (1−1)2+(1−0)2=1(1 - 1)^2 + (1 - 0)^2 = 1. Squared distance to cBc_B is (1−1)2+(1−4)2=9(1 - 1)^2 + (1 - 4)^2 = 9. One against nine picks AA. That one against nine is the diagram headline: two dots per class, two crosses for prototypes, one triangle query closer to the lower cross.

Why more shots help less over time

Averaging nn samples shrinks noise roughly with the square root of nn. Moving from one to five shots steadies the prototype a lot. Moving from twenty to one hundred barely moves it once it sits near the true center. Standard tests therefore average over many random support draws instead of trusting one lucky triple.

Code

Rebuild the worked example with plain Python and confirm distances 1.01.0 against 9.09.0.

def mean_point(pts: list[list[float]]) -> list[float]:    return [sum(p[0] for p in pts) / len(pts), sum(p[1] for p in pts) / len(pts)]

def sq_dist(a: list[float], b: list[float]) -> float:    return (a[0] - b[0]) ** 2 + (a[1] - b[1]) ** 2

support_a: list[list[float]] = [[0.0, 0.0], [2.0, 0.0]]support_b: list[list[float]] = [[0.0, 4.0], [2.0, 4.0]]proto_a: list[float] = mean_point(support_a)proto_b: list[float] = mean_point(support_b)query: list[float] = [1.0, 1.0]
da: float = sq_dist(query, proto_a)db: float = sq_dist(query, proto_b)
print(f"{proto_a} {proto_b}")  # -> [1.0, 0.0] [1.0, 4.0]print(f"{da:.1f} {db:.1f}")  # -> 1.0 9.0

The printout matches the hand arithmetic. Move the query to [1.0,3.0][1.0, 3.0] and the nearer prototype flips to BB by the same two distances swapped.

Watch Out For

Trusting one random support triple

Which three photos land in support moves the prototype. One blurry or odd example drags the average and a good embedding looks weak.

Symptom: accuracy swings widely when the support set is resampled. Fix: average each result over many random support draws as standard nn-shot kk-way tests do, and clean support outliers before averaging.

Expecting far away classes to work as well

An embedding trained on natural photos separates new natural photos well and often fails on scans or satellite views it never saw. Distance means little off the training map.

Symptom: strong results on held out natural classes but near chance on a new sensor. Fix: check domain overlap first in transfer learning, and collect a broader embedding or more shots before shipping.

The Quick Version

  • Few-shot learning labels a new class from a handful of examples instead of hundreds per class.
  • Two supports at 0,0 and 2,0 average to prototype 1,0 in our example.
  • Two supports at 0,4 and 2,4 average to prototype 1,4 in the same space.
  • Query 1,1 sits squared distance 1 from the first prototype and 9 from the second, so it takes the first label.
  • The embedding function carries almost all the accuracy, while the prototype step is only an average plus a nearest lookup.
  • Extra shots help less over time because averaging noise falls with the square root of the count.