Text Classification
Take raw text, turn it into numbers, and pick one label from a fixed list. Spam filters, sentiment meters, and ticket routers are all the same pipeline.
Why Does This Exist?
Inboxes, feeds, review sites, and support queues produce far more text than people can read. Someone must decide what is spam, which reviews praise and which complain, and which department owns each ticket. Hiring readers for all of it does not scale, and hand-written keyword rules rot within weeks as wording drifts.
Text classification answers with one pipeline for all of those jobs: take a span of text, assign it labels from a fixed list, and move on. Sentiment, spam, topic, intent, and routing differ in labels, not in machinery.
This page covers that shared pipeline end to end. The number-making step gets its full treatment in bag of words and TF-IDF, and the workhorse classifier gets its own in logistic regression.
Think of It Like This
A mailroom with labeled bins
A mailroom receives a flood of envelopes and a wall of bins marked with city names. A worker glances at each address and drops it in the matching bin. The worker never rewrites the letter, never summarizes it, just routes it. Spam or inbox, praise or complaint, billing or support: same worker, different bin labels.
Where the picture stops: the worker reads handwriting perfectly and means exactly one bin per letter. Models misread sarcasm and new slang, and some letters need two bins at once, which is the multi-label case the pipeline must be told about upfront.
How It Actually Works
From text to numbers
Classifiers do arithmetic, and strings are not numbers, so the first stage converts. The classical route counts words: bag of words tallies raw frequencies, and TF-IDF down-weights words that appear everywhere so rare telling words count more. The neural route embeds words with similar company near each other, then pools those into one vector per span. Either way the output is a fixed-length vector per text, and everything downstream only ever sees vectors.
The classifier and the softmax
The vector feeds a classifier that scores every candidate label. Logistic regression is the baseline that still wins on small data: fast, legible weights, honest uncertainty. Larger setups use the same scoring head on richer vectors. Raw scores, called logits, can be any size, so the softmax turns them into probabilities that sum to one: exponentiate each score, add the results, divide each by the total. The top probability takes the label. For texts wearing several labels at once, each label gets its own yes-or-no head instead of one shared race.
Worked probabilities
Three labels, logits . Exponentiate: , , . The total is . Divide: , , . They sum to , the first label wins with about four-fifths of the weight, and the gap between first and second logit decides how confident that win looks. The same three numbers appear in the diagram above, sitting on the probability bars.
Code
The softmax decision from the worked example, recomputed so the three probabilities can be checked.
import math
logits = [2.1, 0.4, -0.3]exps = [math.exp(v) for v in logits]total = sum(exps)print([round(v, 3) for v in exps]) # -> [8.166, 1.492, 0.741]print(round(total, 3)) # -> 10.399print([round(v / total, 3) for v in exps]) # -> [0.785, 0.143, 0.071]Exponentiate, total, divide: the first label takes 0.785 and the three shares sum to one.
Watch Out For
Accuracy that flatters an imbalanced task
Symptom: fraud or spam data runs 99 to 1, the model answers the majority class every time, accuracy reads 99 percent, and the classifier is useless. The single number hid the failure because it never asked about the rare class.
Fix: judge with precision, recall, and F1 on the rare class, and set the decision threshold by cost rather than 0.5 (see classification metrics). Report the confusion counts alongside any headline number.
One-label machinery on a multi-label job
Symptom: articles need several tags, the softmax confidently issues exactly one, and recall on second and third tags collapses. Forcing one winner per text was a modeling choice, not a fact about the data.
Fix: switch the head to one yes-or-no output per label with its own threshold, and score with per-label precision and recall. If most texts truly carry one label, keep the race. If co-labels are common, drop it.
The Quick Version
- Text classification maps a span of text to labels from a fixed list through one shared pipeline.
- Text becomes vectors first, by counts like TF-IDF or by learned embeddings, and the classifier only ever sees vectors.
- Logits become probabilities through the softmax, and the top share takes the label.
- Worked case: logits 2.1, 0.4, and minus 0.3 give 0.785, 0.143, and 0.071.
- Judge imbalanced tasks with precision, recall, and F1, never accuracy alone.
- Multi-label jobs need one head per label, not one shared race.