Core ML

Matrix Factorization

Squeezing a massive, mostly-empty grid of users and items into two tiny matrices. By doing this, the algorithm is forced to invent 'hidden topics' (Latent Factors) that explain why a user likes a specific item.

Why Does This Exist?

Classical Collaborative Filtering has a fatal scaling problem.

If Netflix has 100 million users and 10,000 movies, the User-Item matrix has 1 Trillion cells. Worse, 99.9% of those cells are completely empty because the average user has only watched 50 movies. Memorizing this massive, sparse grid requires hundreds of gigabytes of RAM, and calculating the similarity between users takes hours.

In 2006, during the $1 Million Netflix Prize competition, researchers popularized Matrix Factorization (MF) to solve this. Instead of memorizing the massive 1-Trillion-cell matrix, they used linear algebra to compress it into two tiny matrices. This compression forced the algorithm to discover the "hidden reasons" (Latent Factors) why people watch movies, changing the industry forever.

Think of It Like This

Imagine trying to memorize exactly how much 100 different people like 10,000 different songs. It's impossible.

But what if you realize that all music can be described by just 3 "Hidden Factors":

How loud is it?
How fast is it?
Does it have vocals?

Now, you don't need to memorize 10,000 songs. You just give every song a score for those 3 factors. And you give every person a score for how much they like those 3 factors.

If John loves loud, fast music, and "Master of Puppets" is scored as highly loud and fast, you know John will like it without having to memorize the song itself. Matrix Factorization automatically discovers these "Hidden Factors".

How It Actually Works

In linear algebra, any matrix $R$ can be factored (split) into two smaller matrices: $U$ and $V$ . $R \approx U \times V^T$

The Matrices

$R$ (The Rating Matrix): The massive, mostly empty grid of 100M users x 10,000 movies.
$U$ (The User Matrix): A tall, thin matrix of 100M users x $K$ latent factors.
$V$ (The Item Matrix): A tall, thin matrix of 10,000 movies x $K$ latent factors.

$K$ is a number you choose, usually between 10 and 100. It represents the number of "Hidden Factors" (Latent Factors).

The Magic of Latent Factors

The algorithm doesn't know what "Comedy" or "Action" is. But as it trains (usually using Gradient Descent or Alternating Least Squares), it might learn that:

Latent Factor #1 spikes for movies with explosions. (The algorithm invented the concept of "Action").
Latent Factor #2 spikes for movies with lots of crying. (The algorithm invented the concept of "Drama").

Now, every movie is represented as a dense array of $K$ numbers. Every user is also represented as a dense array of $K$ numbers. This array is called an Embedding.

Predicting a Rating

To predict how much User A will like Movie B, you simply take the Dot Product of their embeddings. $\text{Prediction} = \text{User A Embedding} \cdot \text{Movie B Embedding}$ If the user loves Action (high Factor #1), and the movie is an Action movie (high Factor #1), the dot product will be massive, resulting in a high recommended rating.

Show Me the Code

You can use the Surprise library in Python to perform Singular Value Decomposition (SVD), which is the most famous Matrix Factorization algorithm for recommendations.

1from surprise import SVD, Dataset2from surprise.model_selection import cross_validate3
4# 1. Load the built-in MovieLens 100k dataset5# (100,000 real movie ratings from 1,000 users on 1,700 movies)6data = Dataset.load_builtin('ml-100k')7
8# 2. Initialize the SVD (Matrix Factorization) algorithm9# n_factors = K (The number of latent hidden factors to invent, e.g., 50)10algo = SVD(n_factors=50, n_epochs=20)11
12# 3. Train the model and evaluate it using 5-Fold Cross Validation13results = cross_validate(algo, data, measures=['RMSE', 'MAE'], cv=5, verbose=True)14
15# 4. Predict a specific rating16# How much will User '196' like Item (Movie) '302'?17uid = str(196) 18iid = str(302) 19prediction = algo.predict(uid, iid)20print(f"\nPredicted rating for User {uid} on Movie {iid}: {prediction.est:.2f} stars")

Watch Out For

The Cold Start Problem. Because Matrix Factorization relies entirely on the historical interaction matrix ( $R$ ), it suffers from the exact same Cold Start problem as Collaborative Filtering. If a new user signs up, their row in $R$ is completely empty. The algorithm cannot calculate an embedding for them, so it cannot recommend anything.

The Quick Version

The raw User-Item matrix is too massive and too sparse (empty) to use directly.
Matrix Factorization uses linear algebra to compress the massive matrix into two tiny matrices: a User matrix and an Item matrix.
This forces the algorithm to invent Latent Factors (hidden concepts like genre, tone, or speed).
Users and Items are now represented as dense vectors (Embeddings).
To recommend an item, you simply calculate the dot product between the User's embedding and the Item's embedding.

Related concepts

Implicit Feedback