Text Classification

Take a span of text and pick labels from a fixed list. Spam, sentiment, topic and ticket routing are one pipeline with different bin names.

A sentence vector scores three labels as 1.8, 0.2 and minus 0.6, and the softmax turns them into 0.774, 0.156 and 0.070 so the first label wins.

Why Does This Exist?

Inboxes, feeds, review sites and support queues produce far more text than people can read. Someone must decide what is spam, which review praises and which complains, and which team owns each ticket. Hiring readers for all of it does not scale, and hand written keyword rules rot within weeks as wording drifts.

Text classification answers with one pipeline for all of those jobs. Take a span of text, assign labels from a fixed list, and move on. Sentiment, spam, topic, intent and routing differ in labels, not in machinery.

This page covers that shared pipeline end to end. The number making step gets its full treatment in bag of words and the workhorse classifier in logistic regression. The closest application has its own page in sentiment analysis.

The one idea you need first

A classifier only does arithmetic, so text must become numbers first. A span becomes one fixed length vector, by counts or by learned embeddings, and everything after that only sees vectors. If that move feels new, read sentence embeddings for the richer version, then come back.

Think of It Like This

A rail yard that switches each car to a fixed track

You run a small rail yard with three exit tracks marked Billing, Account and Hardware. Each incoming car carries a waybill, and your job is to read it fast and throw the switch so the car rolls to the right track. You never rebuild the car. You route it.

A text classifier is that switch operator. The waybill is the vector, the switch is the scoring head, and the tracks are the fixed labels. Most cars belong on exactly one track, which is the single label case.

Where it stops: real cars carry one load while texts often carry two. A message can ask for a refund and report a broken screen in one breath. The yard needs a new rule for that, which is the multi label case the pipeline must be told about upfront.

How It Actually Works

From text to numbers

Strings are not numbers, so the first stage converts. The classical route counts words and down weights words that appear everywhere, so rare telling words count more. The neural route embeds words that keep similar company near each other, then pools those into one vector per span. Either way the output is one fixed length vector per text.

Call that vector $x$ . It is the only thing the classifier reads.

Scores become shares

The vector feeds a head that scores every candidate label. Call the raw scores logits $z_1, z_2, \dots, z_K$ . Here $z_i$ is the score for label $i$ and $K$ is the number of labels. Logits can be any size, so the softmax turns them into shares that sum to one. Exponentiate each score, add the results, divide each by the total:

p_i = \frac{e^{z_i}}{\sum_j e^{z_j}}

Here $e^{z_i}$ means the exponential of score $i$ , and $p_i$ is the share for label $i$ . The top share takes the label. For texts wearing several labels at once, each label gets its own yes or no head instead of one shared race.

The trade is plain. One shared race is simple and sharp when each text carries one label, and it fails by design when co labels are common. One head per label handles overlap and costs more thresholds to set.

Worked probabilities

Three labels, logits $[1.8, 0.2, -0.6]$ . Exponentiate: $e^{1.8} \approx 6.050$ , $e^{0.2} \approx 1.221$ , $e^{-0.6} \approx 0.549$ . The total is $6.050 + 1.221 + 0.549 = 7.820$ . Divide: $6.050 / 7.820 \approx 0.774$ , $1.221 / 7.820 \approx 0.156$ , $0.549 / 7.820 \approx 0.070$ . They sum to $1.000$ , the first label wins with about three quarters of the weight, and the gap between the first and second logit decides how confident that win looks. The same three shares sit on the bars in the diagram above.

Code

The softmax decision from the worked example, recomputed so the three shares can be checked.

1import math2
3
4def softmax(logits: list[float]) -> list[float]:5    exps: list[float] = [math.exp(v) for v in logits]6    total: float = sum(exps)7    return [v / total for v in exps]8
9
10logits: list[float] = [1.8, 0.2, -0.6]11probs: list[float] = softmax(logits)12print([round(v, 3) for v in probs])  # -> [0.774, 0.156, 0.07]13print(round(sum(probs), 3))  # -> 1.0

Exponentiate, total, divide. The first label takes 0.774 and the three shares sum to one.

Watch Out For

Accuracy that flatters an imbalanced task

Symptom: spam data runs 99 to 1, the model answers the majority class every time, accuracy reads 99 percent, and the filter is useless. The single number hid the failure because it never asked about the rare class.

Fix: judge with precision, recall and F1 on the rare class, and set the decision threshold by cost rather than 0.5. Report the raw counts beside any headline number.

One label machinery on a multi label job

Symptom: articles need several tags, the softmax issues exactly one winner, and recall on second and third tags collapses. Forcing one winner per text was a modeling choice, not a fact about the data.

Fix: switch the head to one yes or no output per label with its own threshold, and score with per label precision and recall. If most texts truly carry one label, keep the race. If co labels are common, drop it.

The Quick Version

Text classification maps a span of text to labels from a fixed list through one shared pipeline.
Text becomes vectors first, by counts or by learned embeddings, and the classifier only sees vectors.
Logits become shares through the softmax, and the top share takes the label.
Worked case: logits 1.8, 0.2 and minus 0.6 give 0.774, 0.156 and 0.070.
Judge skewed tasks with precision, recall and F1, never accuracy alone.
Multi label jobs need one head per label, not one shared race.