Relation Extraction

Once you know the names in a sentence, relation extraction draws the labeled arrow between them, turning words into a triple a database can store.

Two typed mentions in one sentence yield one triple, with born_in scoring 0.665 against lives_in at 0.245 and none at 0.090.

Why Does This Exist?

Knowing the names is not knowing the story. A tagger can tell you that Barack Obama is a person and Hawaii is a place, but a search engine cannot answer who was born where until some system names the arrow between them. That missing arrow is the whole job here.

At web scale the gap hurts. Millions of news sentences mention people and companies every day, and a list of isolated names cannot populate a knowledge graph, answer a precise question, or flag that a drug treats a disease. Relation extraction turns each sentence with two typed mentions into a structured fact, usually a triple of subject, relation, and object, that downstream tables and graphs can query.

This page covers the sentence level case: one sentence, two marked spans, one label from a fixed set plus none. The prerequisite is named entity recognition, which finds the spans and their types. The parent pipeline is information extraction, which chains entities, relations, and events into records. Document level links across sentences need coreference resolution, and that is a separate problem.

The one idea you need first

A typed mention is a span plus a label, such as Barack Obama as PER or Hawaii as LOC. A relation is a labeled link between two mentions, such as born_in between a person and a place. Write a fact as a triple: (Barack Obama, born_in, Hawaii). If either span is wrong, the triple is wrong, which is why the tagger matters so much. The full treatment of spans and BIO tags is in named entity recognition.

Think of It Like This

Strings on a detective board

You pin two photos to a corkboard: one labeled person, one labeled place. A pile of loose photos tells you nothing. Then you pin a string between them and clip on a tag that says born in. Now the board states a fact you can read at a glance.

The photos are the entity mentions, the string is the candidate pair, and the tag is the relation label. A tag that says none means you take the string down. The board holds only when both photos are correct and the tag matches the sentence on the table below.

Where it stops: a real board lets you draw any string you like. This task lets you pick only from a fixed list of relations plus none, so anything outside that list cannot be expressed.

How It Actually Works

Fix a sentence $x$ with two marked spans $s_1$ and $s_2$ and their types $t_1$ and $t_2$ . Fix a relation set $R$ such as born_in, lives_in, works_for, plus a none label for no relation. The classifier reads $x$ with the spans marked and outputs one distribution $p$ over $R$ plus none. In plain words: $x$ is the sentence text, $s_1$ and $s_2$ are the two spans being judged, $t_1$ and $t_2$ are their entity types, $R$ is the allowed vocabulary of links, and $p$ is the predicted probability per link.

Mark the pair

Encoders need to know which two mentions are under judgment, because one sentence holds many pairs. The standard trick wraps them with marker tokens: [E1] Barack Obama [/E1] was born in [E2] Hawaii [/E2]. A transformer reads the whole marked sentence and returns a vector per token. The vectors at the [E1] and [E2] positions, call them $h_1$ and $h_2$ , become the pair representation. Concatenating them preserves order, so (person, place) and (place, person) do not look identical.

Score the relation

A linear layer maps the pair vector to one score per relation, called logits $z$ . With weight matrix $W$ and bias $b$ :

z = W [h_1; h_2] + b

Softmax turns $z$ into probabilities that sum to 1. The predicted relation is the argmax of $p$ . Training is cross entropy against the gold label, so the model pays most when it is confident and wrong.

Worked example

Take the sentence above with $R$ reduced to three labels: born_in, lives_in, none. Suppose the head outputs logits $z = [2.0, 1.0, 0.0]$ .

Exponentiate: $e^{2.0} \approx 7.389$ , $e^{1.0} \approx 2.718$ , $e^{0.0} = 1.000$ . Sum to $11.107$ . Divide:

p = [7.389 / 11.107,\ 2.718 / 11.107,\ 1.000 / 11.107] \approx [0.665,\ 0.245,\ 0.090]

The model predicts born_in at 0.665, keeps lives_in as a plausible second at 0.245, and assigns 0.090 to none. The emitted triple is (Barack Obama, born_in, Hawaii). Change the sentence to visited and the same arithmetic should shift mass toward none, which is exactly the behavior you test after training.

Code

This snippet is the check, not the model. It replays the softmax from the worked example so you can confirm the three probabilities sum to 1 and the argmax picks born_in.

1import math2from typing import List3
4
5def softmax(logits: List[float]) -> List[float]:6    exps = [math.exp(v) for v in logits]  # unnormalized mass per label7    total = sum(exps)8    return [v / total for v in exps]9
10
11labels = ["born_in", "lives_in", "none"]12logits = [2.0, 1.0, 0.0]13probs = softmax(logits)14
15for name, p in zip(labels, probs):16    print(f"{name:<8} {p:.3f}")17# -> born_in  0.66518# -> lives_in 0.24519# -> none     0.09020
21best = labels[max(range(len(probs)), key=lambda i: probs[i])]22print(f"triple: (Barack Obama, {best}, Hawaii)")23# -> triple: (Barack Obama, born_in, Hawaii)

The printout matches the hand calculation to three decimals, and the probabilities sum to 1.000 within rounding. Swap in your own logits from a trained head and this same function tells you whether the winner actually changed.

Watch Out For

Trusting the relation when the spans are wrong

Pipeline systems tag entities first and classify second, so a missed or mistyped span can never recover. If the tagger labels Hawaii as an organization or drops Barack Obama entirely, the classifier still outputs a confident distribution over a broken pair. The symptom is a triple that looks clean but points at the wrong text.

Check span quality before tuning the relation head. Measure entity F1 on your data, inspect ten errors by hand, and switch to a joint model that predicts spans and relations together when cascading mistakes dominate.

Expecting one sentence to cover a document fact

Standard classifiers read a single sentence, but real facts sprawl. A biography introduces a person on page one and states the birthplace three paragraphs later with a pronoun in between. A sentence model sees each piece alone and outputs none everywhere.

Treat cross sentence links as a different task. Resolve mentions with coreference resolution first, then use a document level extractor that can attend across sentences instead of forcing the sentence model to guess.

The Quick Version

Relation extraction picks one label from a fixed set plus none for a marked pair of entities in one sentence.
Marker tokens tell the encoder which pair is under judgment, and the [E1] and [E2] vectors form the pair representation.
A linear layer plus softmax gives probabilities that sum to 1, and the argmax becomes the triple.
Logits [2.0, 1.0, 0.0] give [0.665, 0.245, 0.090], so born_in wins for Barack Obama was born in Hawaii.
Broken spans break triples, and single sentence models cannot link facts spread across a document.