Tokenization
Cutting text into the discrete units a model actually reads, which are usually word fragments rather than words.
Three levels. Word-level splits on whitespace and punctuation, so the vocabulary grows with the corpus and never covers it. Character-level has no unknown tokens and makes sequences five or six times longer. Subword sits between them and won.
Subword schemes — byte pair encoding, WordPiece, SentencePiece — learn frequent fragments from a corpus and decompose anything unseen into pieces they know, so tokenising becomes token plus ising. That's the whole answer to the out-of-vocabulary problem.
The practical consequence people miss: token count is not word count. Merges learned on mostly-English prose mean an unfamiliar script pays more per character, so the same paragraph can cost double in translation, and JSON spends tokens on braces. Pin the tokeniser version with the model, because a minor release changing normalisation changes every sequence.