Skip to content
AI360XpertBeta
Glossary
Definition

GPT

A decoder-only, causally-masked transformer trained on one objective: predict the next token, everywhere, in any text.

Think of It Like This

Like a speed-reader who finishes sentences by guessing the single most logical next word based on every word that came earlier in the book.

GPT stacks causally-masked transformer blocks, so every position can attend to itself and everything before it, never anything after. It trains on a single unsupervised objective, next-token prediction, which scales to nearly unlimited unlabeled text and turns out to implicitly teach grammar, facts, and reasoning patterns along the way.

At inference, the same forward pass runs one step at a time: predict a token, append it, feed the sequence back in, repeat.