Encoder-Decoder Architectures
The three ways to arrange transformer blocks — encoder-only for unrestricted understanding, decoder-only with causal masking for generation, and encoder-decoder running both with cross-attention between them.
Same transformer block, three skeletons. Encoder-only stacks have no causal mask and build a representation of a complete input — BERT's lineage. Decoder-only stacks mask every position causally and generate one token at a time — GPT's lineage. Encoder-decoder runs both, with the decoder cross-attending into the encoder's output, the original design built for translation.
Decoder-only became the dominant large-model choice because next-token prediction scales to unlabeled text in a way paired input-output data never could.