Skip to content
AI360XpertBeta

Graph Neural Networks

Each node collects information from its neighbors, combines it with its own, and repeats, so a network can learn from relationships, not just isolated rows.

A center at 1.0 and 0.0 averages three neighbors to 1.0 and 0.667 and updates to 1.0 and 0.5, and a second layer pulls in two-hop friends
A center at 1.0 and 0.0 averages three neighbors to 1.0 and 0.667 and updates to 1.0 and 0.5, and a second layer pulls in two-hop friends

Why Does This Exist?

A social platform scores accounts for bot behavior. Each account has its own features: post rate, account age, profile completeness. A row-by-row classifier gets most accounts right, then misses one coordinated cluster: a hundred accounts that each look plausible alone but follow each other, share the same links, and joined within the same hour.

That pattern lives in the connections, not in any single row. A model that scores rows independently never sees it. What is missing is the graph itself, who links to whom, plus a rule that lets each account's score depend on its neighbors.

A graph neural network folds the links into the model. Each node builds its state from its own features plus a summary of its neighbors, and stacking that step carries information two or three hops out. The static version of this idea, walks turned into vectors, is graph representation learning. The step-by-step picture is graph neural networks.

Think of It Like This

Neighbors describing one house

An appraiser prices one house. The house itself matters: size, age, condition. But the three nearest sales matter just as much, because buyers pay for the street, not only the bricks.

So the appraiser averages the three neighbors, then blends that average with the house itself. Next round, each neighbor's price already reflects its own neighbors, so the house quietly absorbs two streets away. That is message passing: average around, blend with self, repeat.

Where it stops: houses keep their address no matter how many rounds pass. Node vectors do not. Average too many rounds and every house price converges toward the town average, which is exactly the over-smoothing failure below.

How It Actually Works

One layer is one round of messages. Every node gathers its neighbors' vectors, summarizes them into one vector, and merges that summary with its own state:

hv(k+1)=combine(hv(k),aggregate({hu(k):u∈N(v)}))h_v^{(k+1)} = \mathrm{combine}\big(h_v^{(k)}, \mathrm{aggregate}(\lbrace h_u^{(k)} : u \in \mathcal{N}(v) \rbrace)\big)

Here hv(k)h_v^{(k)} is node vv's vector after kk layers, N(v)\mathcal{N}(v) is its neighbor set, aggregate\mathrm{aggregate} handles any neighbor count with a sum, mean, or max, and combine\mathrm{combine} merges the neighbor summary with the node's own state. Layer zero hv(0)h_v^{(0)} is the raw input features.

Worked example: center 1.0 and 0.0 with three neighbors

Center c=[1.0,0.0]c = [1.0, 0.0]. Neighbors n1=[3.0,0.0]n_1 = [3.0, 0.0], n2=[1.0,2.0]n_2 = [1.0, 2.0], n3=[−1.0,0.0]n_3 = [-1.0, 0.0]. Mean aggregation sums each coordinate and divides by three:

(3.0+1.0−1.0)/3=1.0,(0.0+2.0+0.0)/3=0.667(3.0 + 1.0 - 1.0) / 3 = 1.0, \quad (0.0 + 2.0 + 0.0) / 3 = 0.667

So the neighbor summary is [1.0,0.667][1.0, 0.667]. Merging with self by averaging all four vectors gives:

(1.0+3.0+1.0−1.0)/4=1.0,(0.0+0.0+2.0+0.0)/4=0.5(1.0 + 3.0 + 1.0 - 1.0) / 4 = 1.0, \quad (0.0 + 0.0 + 2.0 + 0.0) / 4 = 0.5

The new center is [1.0,0.5][1.0, 0.5]. The xx coordinate stays 1.0 because the neighbors balance, while the yy coordinate rises toward the one neighbor carrying signal. The diagram shows these exact four dots, the mean arrow, and the updated center.

Why the three famous variants differ

A Graph Convolutional Network averages with degree normalization so high-degree hubs do not dominate. GraphSAGE samples a fixed neighbor count per node, which caps cost on graphs where one node has thousands of links. A Graph Attention Network learns a weight per neighbor instead of fixed averaging, so each node decides whom to listen to. All three use the same worked update above and differ only in the aggregate\mathrm{aggregate} step. The contrastive way to train such vectors without labels is contrastive learning, and the per-pair weighting idea comes from attention mechanism.

Code

The point is the check: rebuild the worked aggregation and confirm the summary reads 1.000 and 0.667 with the update at 1.000 and 0.500.

center: list[float] = [1.0, 0.0]neighbors: list[list[float]] = [[3.0, 0.0], [1.0, 2.0], [-1.0, 0.0]]
agg: list[float] = [sum(n[d] for n in neighbors) / len(neighbors) for d in range(2)]new: list[float] = [(center[d] + sum(n[d] for n in neighbors)) / (len(neighbors) + 1) for d in range(2)]
print([f"{v:.3f}" for v in agg])  # -> ['1.000', '0.667']print([f"{v:.3f}" for v in new])  # -> ['1.000', '0.500']

Change n2n_2 from [1.0,2.0][1.0, 2.0] to [1.0,0.0][1.0, 0.0] and both outputs collapse to [1.0,0.0][1.0, 0.0]. The update only carries what the neighbors actually contain.

Watch Out For

Stacking layers the way a vision net stacks them

Symptom: depth goes from two layers to six and accuracy falls instead of rising, with node vectors looking more alike each layer.

Fix: sweep depth as its own small search around two to three layers and stop there. Each layer is an averaging round, and past that radius every node blends toward the same value. Depth here buys reach, not expressiveness.

Aggregating full neighborhoods on a hub-heavy graph

Symptom: training time swings wildly by batch because one sampled node has thousands of neighbors and its full aggregation dominates the step.

Fix: sample a fixed neighbor count per node, the GraphSAGE rule, so per-node cost stays capped no matter the true degree. Keep full aggregation for small graphs only.

The Quick Version

  • A graph neural network layer averages neighbor vectors and merges the result with the node's own vector.
  • Our center at 1.0 and 0.0 with neighbors 3.0 and 0.0, 1.0 and 2.0, minus 1.0 and 0.0 gives neighbor mean 1.0 and 0.667.
  • Including the center itself, the updated vector is 1.0 and 0.5.
  • GCN normalizes by degree, GraphSAGE samples neighbors for scale, and GAT learns per-neighbor weights.
  • Past three or four layers, repeated averaging erases the distinctions the model needs, so practical nets stay shallow.