Coreference Resolution

Coreference resolution identifies all expressions in a text that refer to the same entity, so a model knows that "he", "the CEO", and "Tim Cook" are the same person.

Why Does This Exist?

When humans communicate, whether in written text or spoken language, we naturally and effortlessly rely on a system of linguistic shortcuts. Instead of endlessly repeating the full name of a person, a place, or an object every single time we refer to it, we use pronouns (like "she", "he", "it", or "they"), definite noun phrases (like "the company" or "the organization"), and other referring expressions. For example, consider the sentence: "Jane wanted to buy a new car, but she didn't have enough money for it, so she asked her parents for a loan." A human reader instantly understands that "she" and "her" refer to Jane, "it" refers to the new car, and "parents" are Jane's parents.

However, for a computer algorithm or a machine learning model, tracking these connections across sentences and paragraphs is incredibly difficult. Without a mechanism to map these different textual expressions (called mentions) back to the same underlying real-world entity, a Natural Language Processing (NLP) pipeline treats "Jane", "she", and "her" as completely unrelated concepts. This profound limitation cripples the system's ability to extract meaningful information, summarize text, translate languages accurately, or answer questions. If a question-answering model cannot figure out who "she" is, it cannot correctly answer "Who asked for a loan?".

This is exactly why coreference resolution exists. It is the NLP task of finding all expressions in a text that refer to the same entity and linking them together into what is known as a coreference chain or cluster. By successfully resolving these coreferences, an AI model builds a coherent and unified representation of the entities being discussed, which is an absolutely essential prerequisite for higher-level semantic understanding and reasoning. It bridges the gap between the surface-level string of words and the underlying meaning of the narrative.

Think of It Like This

Following Characters in a Complex Novel

Think of reading a dense, multi-character novel like Game of Thrones. The author introduces a character, say "Daenerys Targaryen", in the first chapter. A few pages later, she might be referred to as "the Mother of Dragons", "she", "her", or "the queen".

As a reader, your brain maintains a mental "dossier" or a file folder for Daenerys. Every time you encounter a new pronoun or nickname that points to her, you don't create a new character in your head; instead, you mentally file that new mention directly into Daenerys's existing folder.

Coreference resolution is exactly this process for an AI. The algorithm reads the text and acts as an incredibly meticulous librarian. It spots mentions in the text, checks if they match any of the open "dossiers" (clusters), and files them accordingly. If a mention doesn't seem to match anyone currently being tracked, the algorithm opens a brand new dossier for a newly introduced entity. The final output is a perfectly organized filing cabinet where all mentions of a single person or object are safely stored in their designated folder.

How It Actually Works

Solving coreference resolution is traditionally framed as a combination of finding the mentions and then deciding which mentions should be clustered together. Modern neural approaches, particularly those powered by transformer architectures, typically break this down into a sequence of steps.

1. Mention Detection

Before the system can link expressions together, it must first identify which spans of text are even eligible to be linked. This step is called mention detection. The model scans the text and extracts potential referring expressions. This includes proper nouns ("Apple Inc."), pronouns ("they", "it"), and noun phrases ("the giant tech company"). In modern end-to-end models, this step is often jointly trained with the coreference step, meaning the model learns to assign a "mention score" to every possible span of text, aggressively filtering out spans that are unlikely to represent entities (like "is going to").

2. Contextual Representation and Embedding

Once potential mentions are identified, the model needs to understand their meaning within the context of the sentence. Using a transformer encoder (like BERT, RoBERTa, or a similar architecture), the text is processed to generate dense vector representations (embeddings) for each word. The representation for a specific mention span is usually constructed by combining the embeddings of the words within that span, sometimes augmented with an attention mechanism that highlights the "head word" of the phrase. This dense vector captures not just what the words are, but how they are being used in that specific grammatical and semantic context.

3. Pairwise Scoring

With contextual embeddings in hand, the core of the coreference resolution process begins: comparing mentions. For every pair of candidate mentions (where the second mention appears after the first in the text), the model calculates a coreference score. This score predicts the probability that these two specific mentions refer to the exact same entity.

The scoring function takes into account several factors:

The similarity of their contextual embeddings.
Distance features (how many words or sentences separate them).
Speaker information (in dialogue, "I" and "you" depend heavily on who is talking).
Syntactic compatibility (e.g., singular pronouns rarely corefer with plural nouns).

Mathematically, the pairwise score $s(i, j)$ between a preceding mention $i$ and a later mention $j$ is typically computed using a feed-forward neural network that concatenates the representation of $i$ , the representation of $j$ , and an element-wise similarity between them:

$s(i, j) = \text{FFNN}([\mathbf{h}_i, \mathbf{h}_j, \mathbf{h}_i \circ \mathbf{h}_j, \phi(i, j)])$

Here, $\mathbf{h}$ represents the mention vectors, $\circ$ denotes element-wise multiplication, and $\phi(i, j)$ represents metadata features like distance.

4. Clustering and Chain Formation

The final step is to take the pairwise scores and group the mentions into coherent clusters (coreference chains). A common approach is the antecedent-based method. For every mention in the document, the system looks at all preceding mentions and selects the one with the highest pairwise coreference score (provided the score is above a certain threshold). If no preceding mention scores high enough, the current mention is assumed to be a new entity.

By linking each mention to its most likely antecedent, a graph is formed. The connected components of this graph become the final coreference clusters. For example, if "he" links to "the CEO", and "the CEO" links to "Tim Cook", they all belong to the single cluster representing that specific person.

Code

While training a state-of-the-art coreference model from scratch is complex, using an existing one is straightforward. We can use the popular fastcoref library (which wraps efficient transformer models) to resolve coreferences in a short text.

1# -> pip install fastcoref2from fastcoref import FCoref3
4# 1. Load the pre-trained coreference model5model = FCoref(device='cpu')6
7text = "Jane wanted to buy a new car. She didn't have enough money for it, so she asked her parents."8
9# 2. Run predictions10preds = model.predict([text])11
12# 3. Extract and print the coreference clusters13clusters = preds[0].get_clusters(as_strings=False)14clusters_strings = preds[0].get_clusters()15
16print("Found clusters:")17for i, cluster in enumerate(clusters_strings):18    print(f"Cluster {i+1}: {cluster}")19    20# -> Found clusters:21# -> Cluster 1: ['Jane', 'She', 'she', 'her']22# -> Cluster 2: ['a new car', 'it']

Watch Out For

Cataphora vs. Anaphora

Anaphora is when a pronoun refers back to a previously mentioned entity (e.g., "John arrived, and he sat down"). This is the most common pattern and models are very good at it. Cataphora is when the pronoun appears before the entity it refers to (e.g., "Before she arrived, Jane called.").

Many models aggressively look backwards for antecedents and struggle significantly with cataphoric references, often either failing to link the pronoun or incorrectly linking it to whatever entity appeared in the preceding sentence, completely destroying the semantic meaning.

Gender Bias in Pre-trained Models

Coreference resolution models are notoriously susceptible to inheriting gender biases from their training data. For example, if given the sentence "The doctor yelled at the nurse because she was late," a biased model might incorrectly resolve "she" to "the nurse" simply because of historical statistical correlations in the training corpus, ignoring syntactic or contextual cues that might point to the doctor.

Always evaluate coreference models on bias benchmarks (like WinoBias) before deploying them in sensitive or user-facing applications.

The Quick Version

The Goal: Link all linguistic expressions (mentions) in a text that refer to the exact same real-world entity into clusters.
The Challenge: Pronouns and ambiguous noun phrases make it impossible for naive keyword matching to track entities across sentences.
Mention Detection: The system first identifies all valid noun phrases and pronouns that could possibly refer to an entity.
Contextual Scoring: A neural network compares pairs of mentions, using transformer embeddings and distance features to calculate the probability that they are referring to the same thing.
Clustering: Mentions are linked to their highest-scoring antecedent, forming chains that represent unified entities throughout the entire document.