Mastering the Game of Go without Human Knowledge (AlphaZero)

The Problem

Artificial intelligence had recently achieved a historic milestone when AlphaGo defeated human champion Lee Sedol. However, that original AlphaGo system was still heavily reliant on human expertise. It was initialized by training on thousands of human expert games to bootstrap its knowledge. Relying on human data not only limits the system to human-level creativity but also restricts the application of such AI to domains where vast amounts of high-quality human data are available.

The Idea

The researchers at DeepMind asked: can an algorithm master a domain as complex as Go completely from scratch? AlphaGo Zero (and subsequently AlphaZero) removed all human data, handcrafted features, and human-specific heuristics. Instead, it started with only the basic rules of the game and learned entirely through self-play—playing millions of games against itself, continuously improving its own evaluations and strategies based purely on the win/loss outcomes.

How It Works

The system elegantly combines a neural network with Monte Carlo Tree Search (MCTS) in a continuous loop of self-improvement.

• Single Neural Network: Unlike earlier versions that used separate policy and value networks, AlphaZero uses a single deep ResNet that takes the board state as input and outputs both a policy (which moves to consider) and a value (who is likely to win).
• Self-Play with MCTS: During self-play, MCTS uses the neural network to guide its search, prioritizing promising moves. The result of the MCTS provides a stronger, more refined policy than the raw network output.
• Continuous Training: The network is continuously retrained to predict the improved MCTS policy and the actual winner of the self-play game. As the network improves, the MCTS becomes stronger, which in turn generates higher-quality data for the next round of training.

Why It Mattered

AlphaZero proved that human data is not a prerequisite for achieving superhuman intelligence in perfectly observable, deterministic environments. In fact, by discarding human knowledge, AlphaZero learned faster, achieved a much higher Elo rating, and discovered entirely novel, unconventional strategies that human players had never conceived in thousands of years of Go history.

What Came After

The generalized AlphaZero algorithm was subsequently applied to Chess and Shogi, mastering both in a matter of hours. The underlying principles of powerful self-play and MCTS guided by neural networks influenced later breakthroughs like MuZero, which can master games without even being given the rules, and AlphaFold, which revolutionized computational biology by predicting protein structures.