Perceptron
The perceptron is the simplest neural network. See how one neuron draws a line, and why the XOR problem forces it to grow a hidden layer.
A single neuron takes inputs, multiplies them by weights, adds them up, and outputs a binary decision through a threshold function. It's the simplest possible decision boundary.
Linear Boundary
Because it only uses a simple weighted sum, the decision boundary it draws in space is always perfectly straight. It separates the world into two halves.
The XOR Problem
Exclusive-OR places data in opposite corners. No matter how you angle a straight line, it will always leave at least one point on the wrong side. The single neuron fails completely here.
Where It Breaks
The only way to solve this is to add a hidden layer. The hidden neurons create new features by combining the original inputs, allowing the final layer to draw a non-linear boundary in the original space.
The Quick Version
- A perceptron is the simplest network: weighted sum plus threshold.
- It can only draw a straight line boundary.
- The XOR problem is impossible for a single straight line.
- Adding a hidden layer provides the depth needed to bend the boundary.