CRAFT Finds Text by Characters

CRAFT scores each character and the gaps between them, then links nearby pieces into words that bend and tilt.

CRAFT paints character blobs at 0.88 and affinity bridges at 0.76, linking three curved letters into one word box

Why Does This Exist?

Whole-word boxes assume straight lines. Curved bottle text and wavy signs break that shape, and one rigid rectangle cannot follow the bend.

CRAFT goes bottom-up from characters. It extends text detection, hands cleaner polygons to optical character recognition, and splits the line problem that text recognition then reads.

Think of It Like This

Beads strung on a curved wire

Beads sit along a bent wire with short threads between neighbours. Each bead is a character and each thread is the affinity saying these two belong together.

It fails when threads snap in glare. Loose beads roll apart and the word splits even though every character was found.

How It Actually Works

Predict region score $r$ per pixel for character centers, where $r$ near 1 means a character. Predict affinity $a$ for gaps between characters, where $a$ near 1 means link. Keep pixels with $r > 0.5$ or $a > 0.4$ , then join connected blobs into polygons.

Worked example

Three curved letters return $r = 0.88, 0.91, 0.85$ with bridges $a = 0.76, 0.72$ . Both bridges pass 0.4, so all three join into one polygon with IoU 0.86. A whole-word box on the same curve scores 0.58 because its straight edges cut the bend. The 0.28 gap is the curve premium.

Watch Out For

Setting the affinity threshold too high

A strict 0.7 link threshold splits spaced fonts into single letters. The symptom is perfect characters but shattered words. Lower affinity to 0.4 before touching region threshold, since links are naturally weaker than centers.

The Quick Version

CRAFT predicts character regions $r$ plus inter-character affinity $a$ .
Fixture blobs 0.88 with bridges 0.76 join three bent letters at IoU 0.86.
Bottom-up linking follows curves that rigid boxes miss by 0.28.
Characters are easier to find across fonts than whole words.
Tune affinity first when words shatter into letters.