Graph Representation Learning
A graph is links, but models want rows of numbers. This page turns each node into a short vector so linked nodes land near each other and plain models can use them.
Why Does This Exist?
A fraud team holds a network of accounts and payments. They want two answers: which nodes are fake, and which missing links should exist. Standard classifiers want flat rows with fixed columns, but node A has five friends and node B has five thousand. There is no fixed-column version of a neighborhood.
Graph representation learning bridges that gap. It maps every node to a fixed-length dense vector, an embedding, such that nodes close in the graph end up close in vector space. Once each node is a vector, plain tools work again: a classifier for node scores, cosine similarity for link candidates, clustering for communities. The message-passing way to learn those vectors is graph neural networks, and the no-label training objective behind many of them is contrastive learning.
Think of It Like This
Tourists mapping the subway by wandering
A visitor cannot read the subway map, so they ride at random and write down the stations they pass: Waterloo, Embankment, Charing Cross, Piccadilly. Thousands of tourists do the same. Stations that share a line keep appearing next to each other in these notebooks, while far-apart stations almost never do.
A text model then reads the notebooks and gives each station coordinates, placing frequent neighbors near each other. The map becomes lists of numbers that still remember the tracks.
Where it stops: tourists only record which stations sit together, never why. Two hub stations with identical roles on different lines look unrelated in the notebooks even though they do the same job, which is the role-versus-community trap below.
How It Actually Works
The goal is one sentence: nodes that share structure in the graph get vectors that sit near each other. Closeness in the graph becomes closeness by dot product or cosine in vector space.
Worked example: linked pair at 0.998, distant pair at minus 0.937
Give four accounts two-number vectors. Linked pair and . Distant node . Cosine similarity is the dot product divided by both lengths:
because . Against the distant node:
because . Linked nodes sit almost on top of each other while the distant node points the opposite way. That gap is what lets similarity search recommend links and classifiers separate fakes. The diagram draws these four vectors with the two cosine labels.
How walks make sentences from a graph
DeepWalk starts at a node, picks a random edge, walks about ten steps, and records the path as a sentence. Repeat from every node and train a word-vector model on the sentences, so nodes sharing walk contexts get nearby vectors. Node2Vec adds return weight and in-out weight : small keeps the walker near home and captures community, small pushes it outward and captures structural role. Both are transductive lookup tables: a node that joins tomorrow needs a fresh run. The inductive fix that learns a function over features instead is graph neural networks, which is also why recommender graphs often pair embeddings with collaborative filtering.
Code
The point is the check: rebuild both cosine scores from the worked vectors and confirm 0.998 against minus 0.937.
import math
a: list[float] = [0.9, 0.1]b: list[float] = [0.85, 0.15]c: list[float] = [-0.8, 0.2]
def cosine(u: list[float], v: list[float]) -> float: dot: float = u[0] * v[0] + u[1] * v[1] nu: float = math.sqrt(u[0] * u[0] + u[1] * u[1]) nv: float = math.sqrt(v[0] * v[0] + v[1] * v[1]) return dot / (nu * nv)
print(f"{cosine(a, b):.3f}") # -> 0.998print(f"{cosine(a, c):.3f}") # -> -0.937Swap to and the first print drops to minus 0.937 as well. Closeness follows the graph, not the coordinate accident.
Watch Out For
Reading every nearby pair as the same community
Symptom: embeddings cluster two CEO hubs from different companies together with true friend groups, and a role classifier built on community-tuned walks misses every hub.
Fix: decide before training whether similar means same friends or same role, then tune the walker. DeepWalk defaults to community. Push Node2Vec outward for roles, or use a role-specific method, and validate with a probe that separates the two meanings.
Serving a new node with a stale lookup table
Symptom: a node that joined after training has no vector, and the pipeline either errors or falls back to zeros while neighbors score normally.
Fix: treat walk embeddings as a batch artifact with a refresh rule, or switch serving to an inductive function that builds vectors from features and neighborhoods on the fly. Never silently score newcomers as zeros.
The Quick Version
- Graph representation learning maps each node to a fixed dense vector so plain models can read graph structure.
- Our linked pair scores cosine 0.998 while the distant pair scores minus 0.937, which is the gap link prediction needs.
- DeepWalk turns random walks into sentences for a word-vector model, and Node2Vec steers walks with return weight p and in-out weight q.
- Walk tables are transductive and need retraining for new nodes, while message-passing functions handle newcomers from features.
- Similar must be defined up front as shared community or shared role, because one walker setting cannot serve both.