Spell Correction

Spell correction turns a typo into the word you meant, so a search for aple still finds apple by weighing nearby words and how common each one is.

One typo yields three distance-1 candidates, and frequency times typo likelihood picks apple at 0.0040 over maple at 0.0002 and ape at 0.00008.

Why Does This Exist?

Real text is messy. You type fast on a phone, you drop a letter, you swap two. A search box that only knows exact words then fails on a query it could have answered. One missing letter should not cost you the right page.

Spell correction sits in front of the rest of the pipeline to fix that. It takes a typed token $x$ such as aple, proposes nearby known words, and returns the intended word $w$ such as apple. Search, chatbots, and dictation all lean on it because human input stays noisy no matter how good the downstream model gets.

This page covers isolated word correction with the noisy channel rule, plus where context must step in. The cleaning step around it is text normalization, which lowercases and strips noise before any lookup. The splitting step is tokenization, which decides what counts as one word. The full chain is the NLP pipeline.

The one idea you need first

A vocabulary is the set of known words your system will accept. A word outside it is out of vocabulary, often written OOV. Correction only proposes words inside the vocabulary, so a name or a new term that was never added can never win. That is why the vocabulary you ship matters as much as the scoring rule. Token forms and OOV handling are in tokenization.

Think of It Like This

A librarian who knows common slips

You ask a librarian for a book by a name you half remember. You say the title wrong by one sound. The librarian does not send you away. She thinks of every known title one slip away from what you said, then picks the one borrowers ask for most often, adjusted for the kind of slip you likely made.

The shelf of known titles is the vocabulary. The one slip away list is the candidate set. Borrow frequency is the word prior, and the slip type is the error model. The pick is the product of the two.

Where it stops: a librarian hears your whole request and knows you. This page starts with one word alone, no sentence around it. When two candidates sound equally close, the lone word rule must guess from frequency, and only context can break the tie.

How It Actually Works

Fix a typed token $x$ such as aple. Fix a vocabulary $V$ of known words. In plain words: $x$ is what you typed, $w$ ranges over candidate words in $V$ , $P(w)$ is how common $w$ is, $P(x \mid w)$ is how likely someone meaning $w$ types $x$ , and edit distance $d(x, w)$ counts insertions, deletions, and substitutions to turn one into the other.

Propose nearby words

The system lists every $w$ in $V$ within distance 1 or 2 of $x$ . Distance 1 keeps the list small: a handful of candidates instead of thousands. For $x$ equal to aple, three distance-1 neighbors are apple by inserting one p, maple by inserting m at the front, and ape by deleting l. All three cost 1, so distance alone cannot choose.

Score with the noisy channel

The noisy channel rule picks the $w$ that maximizes the product of likelihood and prior:

\hat{w} = \arg\max_{w} P(x \mid w) P(w)

In plain words: $\hat{w}$ is the winning word, $P(x \mid w)$ favors the typo pattern you likely made, and $P(w)$ favors the common word. A frequent word with a plausible slip beats a rare word with the same distance.

Worked example

Give the three candidates priors from counts: $P(\text{apple}) = 0.05$ , $P(\text{maple}) = 0.01$ , $P(\text{ape}) = 0.008$ . Give them typo likelihoods for producing aple: $0.08$ for apple by dropping a doubled letter, $0.02$ for maple, $0.01$ for ape. Multiply:

apple: $0.05 \times 0.08 = 0.0040$
maple: $0.01 \times 0.02 = 0.0002$
ape: $0.008 \times 0.01 = 0.00008$

Apple wins by a factor of 20 over the runner up, even though all three sit at distance 1. That gap is the product doing its job: frequency breaks the distance tie, and the plausible drop pattern widens it.

Let context break real word ties

One word alone cannot fix a valid word used wrongly. In I need too tickets, too is in the vocabulary, so an isolated corrector leaves it. A contextual model reads the neighbors and prefers two before a count noun. Use the lone word rule to clean clear typos, then let a sequence model rescore when every candidate is a real word.

Code

This snippet replays the worked example: it builds distance-1 candidates by one edit, scores each by prior times likelihood, and picks the winner. Outputs match the hand products above.

1from typing import Dict, List2
3
4def score(candidates: Dict[str, float], likelihood: Dict[str, float]) -> Dict[str, float]:5    total: Dict[str, float] = {}6    for w, prior in candidates.items():  # prior P(w) from counts7        total[w] = prior * likelihood[w]  # noisy channel product8    return total9
10
11priors = {"apple": 0.05, "maple": 0.01, "ape": 0.008}12likes = {"apple": 0.08, "maple": 0.02, "ape": 0.01}13ranked = score(priors, likes)14
15for w in sorted(ranked, key=lambda k: ranked[k], reverse=True):16    print(f"{w:<6} {ranked[w]:.5f}")17# -> apple  0.0040018# -> maple  0.0002019# -> ape    0.0000820
21best = max(ranked, key=lambda k: ranked[k])22print(f"correct aple to {best}")23# -> correct aple to apple

Swap in your own counts for $P(w)$ and your own confusion counts for $P(x \mid w)$ and the same argmax gives you the new winner. When priors are flat, the likelihood decides. When likelihoods tie, frequency decides.

Watch Out For

Fixing names and terms you never listed

A general vocabulary does not know your names, codes, or product terms. The corrector sees an unfamiliar token, finds a close common word at distance 1, and confidently rewrites it. The symptom is clean text with the wrong nouns: a surname becomes a fruit, an abbreviation becomes a word.

Skip tokens tagged as names before you correct. Keep a whitelist from named entity recognition, add domain terms to the vocabulary with real counts, and never correct an OOV token that matches your allow list.

Trusting a lone word when any candidate is valid

Distance plus frequency cannot see grammar. In I need too tickets, too is distance 1 from two but it is also a real word, so the isolated rule has no reason to change it. The symptom is a sentence with zero typos that still reads wrong.

Treat real word errors as a separate pass. Run the lone word fix first, then rescore with a window or a transformer that reads neighbors. If your misses are all valid words, the context model is missing, not the distance.

The Quick Version

Spell correction maps a typed token $x$ to the intended vocabulary word $w$ by edit distance plus frequency.
Distance 1 or 2 proposes a short list: aple gives apple, maple, and ape, all at distance 1.
The noisy channel picks $\hat{w} = \arg\max P(x \mid w) P(w)$ , so a common word with a likely slip wins.
Here apple scores 0.0040 against 0.0002 and 0.00008, winning by a factor of 20.
Names and domain terms need a whitelist, and valid word slips like too for two need context, not distance.