Core ML

Two-Tower Retrieval

A neural network that passes the User through one 'tower' and the Item through another. The two towers never touch until the very end, allowing for blazing fast candidate retrieval.

In a Two-Tower model, the User Tower and Item Tower run independently. Their outputs (embeddings) are compared at the very end using a simple dot product.

Why Does This Exist?

In the Recommender System Architecture, the first stage is Retrieval. The goal of Retrieval is to quickly slash a catalog of 10 million items down to 500 items in under 50 milliseconds.

You cannot use a massive, complex neural network for Retrieval. If a neural network tries to compare the User against all 10 million items one-by-one, it will take hours.

The Two-Tower Neural Network (also known as a Dual Encoder) was invented specifically to solve this speed problem. By structurally separating the User data from the Item data, we can pre-calculate 99% of the math overnight, allowing for instant retrieval during the day.

Think of It Like This

Imagine trying to match 1,000 job applicants to 1,000 open jobs.

The Slow Way (Cross-Encoder): You lock an applicant in a room with a job description and ask them to discuss it for 10 minutes. To test every combination, you need $1,000 \times 1,000 = 1,000,000$ interviews. It takes years.

The Fast Way (Two-Tower): You ask every applicant to fill out a personality survey. You separately ask every hiring manager to fill out a job requirement survey. You do this independently. Then, you just overlay the surveys on a computer and instantly find the best matches. This is the Two-Tower approach.

How It Actually Works

The model consists of two entirely separate neural networks (the "Towers").

1. The User Tower

This neural network only looks at User features: their age, their location, and their past clicking behavior. It crunches all this data and outputs a single vector (an Embedding) representing the user. For example, [0.4, -0.2, 0.8].

2. The Item Tower

This neural network only looks at Item features: the video title, the video category, and the upload date. It crunches this data and outputs an Item Embedding. For example, [0.5, -0.1, 0.7].

3. The Dot Product

At the very top of the model, the two towers finally meet. We calculate the dot product (the mathematical similarity) between the User Embedding and the Item Embedding. If the vectors point in the same direction, the score is high, and we recommend the item.

Why is this so fast?

The secret to Two-Tower retrieval is Caching.

Because the Item Tower never looks at the User, the Item Embedding is always exactly the same, regardless of who is asking. Therefore, you don't run the Item Tower in real-time. You run the Item Tower overnight. You calculate the embeddings for all 10 million videos, and you save them into a Vector Database (like FAISS or Pinecone).

When John logs in:

You run the User Tower in real-time to generate John's embedding (takes 5 milliseconds).
You ask the Vector Database: "Find the 500 item embeddings that are closest to John's embedding."
Using Approximate Nearest Neighbor (ANN) search, the database returns the top 500 items instantly (takes 20 milliseconds).

You have just searched 10 million items in 25 milliseconds.

Show Me the Code

In modern TensorFlow, Google provides the tensorflow_recommenders (TFRS) library specifically for building Two-Tower models.

1import tensorflow as tf2import tensorflow_recommenders as tfrs3
4# 1. Define the User Tower5user_model = tf.keras.Sequential([6    tf.keras.layers.StringLookup(vocabulary=unique_user_ids),7    tf.keras.layers.Embedding(len(unique_user_ids) + 1, 32), # 32-dimensional embedding8    tf.keras.layers.Dense(32, activation="relu")9])10
11# 2. Define the Item Tower12item_model = tf.keras.Sequential([13    tf.keras.layers.StringLookup(vocabulary=unique_movie_titles),14    tf.keras.layers.Embedding(len(unique_movie_titles) + 1, 32),15    tf.keras.layers.Dense(32, activation="relu")16])17
18# 3. Define the Two-Tower Retrieval Model19class TwoTowerModel(tfrs.Model):20    def __init__(self, user_model, item_model):21        super().__init__()22        self.user_model = user_model23        self.item_model = item_model24        # The task calculates the dot product between the two towers25        self.task = tfrs.tasks.Retrieval()26
27    def compute_loss(self, features, training=False):28        # Pass user features through User Tower29        user_embeddings = self.user_model(features["user_id"])30        # Pass item features through Item Tower31        item_embeddings = self.item_model(features["movie_title"])32        33        # Calculate the loss (forces matching pairs to have high dot products)34        return self.task(user_embeddings, item_embeddings)

Watch Out For

Late Interaction prevents complex reasoning. Because the two towers cannot talk to each other until the very end, the model cannot learn complex relationships. For example, it cannot easily learn "User A likes Action movies, but only if they star Tom Cruise." This is why Two-Tower models are strictly used for the Retrieval stage. The surviving 500 items are passed to a much heavier Ranking Model (where the features are allowed to mix and interact) to make the final decision.

The Quick Version

Retrieval requires searching millions of items in milliseconds.
Two-Tower Models separate the math: one neural network processes the User, and a completely separate neural network processes the Item.
They meet at the very end via a simple Dot Product.
Because the Item Tower is independent of the user, you can pre-calculate all 10 million Item Embeddings overnight.
In production, you only calculate the User Embedding in real-time, and use a Vector Database to instantly find the closest items.

Related concepts

Learning to Rank (LTR)