Visual explainer

Multi-Layer Perceptron

Stacked neuron layers with bends between them turn straight cuts into curved borders.

One straight cut gets three of four XOR points.

You draw your best line through XOR and one corner always lands wrong. Three of four is your ceiling with one cut. You need depth that bends, not a better straight edge.

Build One Neuron

Each neuron forms 0.8 from weights 0.5 and bias.

You weight inputs 1 and 1 by 0.5 each, add bias minus 0.2, and get 0.8. ReLU keeps 0.8 since it is positive. Every neuron repeats this mix, then bend.

Push Signals Forward

Signals flow 2 to 3 to 1 with ReLU bends.

Two inputs feed three hidden units, then one output. Each arrow carries a weighted share forward while each unit bends its sum. Layer by layer, raw pixels become corner clues.

Bends Stop Collapse

Two straight layers fold into one straight layer.

Skip the bend and layer two just multiplies layer one. Two matrices become one matrix, so depth buys nothing. ReLU breaks the fold and lets borders curve.

Curves Take All

Bent borders take all four XOR points at once.

Score the same four dots on one scale. Straight takes three while bent takes four. The extra point is the payoff your hidden layer bought you.

Depth Versus Width

Go deeper for hierarchies and wider for memorized quirks.

You go deeper when clues stack from edges to shapes. You go wider when you must memorize scattered quirks. Flip to depth first since it reuses parts better.

Where It Breaks

Extra layers memorize noise and starve early gradients.

Add units past the signal and training hits 100 while fresh data sags. Gradients also thin as they travel back through bends. You prune, penalize, and stop early instead.

The Quick Version

One cut caps at three of four.
Weights 0.5 plus bias give 0.8.
Signals flow 2 to 3 to 1.
Straight stacks fold into one.
Bends lift you to four of four.
Depth builds hierarchies, width memorizes.
Too much depth memorizes noise.