System 1 Decision Models
System 1 models process input in a single pass to produce deterministic, typed decisions instead of generating probabilistic text token by token.
Why Does This Exist?
When building agentic applications, you often need to route an incoming request to the right backend system, classify its intent, or extract metadata (like severity). The standard approach has been to prompt a generative LLM to output a JSON object. However, generative models are fundamentally probabilistic text generators. They produce output one token at a time (autoregressively). This decoding loop is extremely slow (often taking several seconds) and highly prone to hallucination, requiring you to write complex parsing logic and retries to handle malformed JSON. We need a way to make fast, typed decisions without generating text.
Think of It Like This
Imagine a sorting facility for mail. A System 2 generative model is like a scribe who reads an envelope, thinks carefully, and then writes a detailed letter describing where the package should go, one word at a time. A System 1 decision model is like a scanner that reads the barcode in a split second and immediately flips a switch to route the package down the correct chute. It doesn't write anything; it just acts.
How It Actually Works
System 1 decision models replace the slow autoregressive decoding step with a fast, deterministic architecture.
1. The Encoder Backbone
Instead of a full encoder-decoder stack, these models use only an encoder (such as ModernBERT or mmBERT). The encoder reads the entire input state (the text, email, or JSON payload) simultaneously and constructs a dense, contextual vector representation of the input.
2. Parallel Decision Heads
On top of this shared representation, the model places multiple distinct "heads". Each head is a small, non-autoregressive neural network layer trained for a specific task. Unlike an LLM that predicts the next word, these heads predict a typed output directly:
- Choice Head: Selects from a predefined enum (e.g.,
Category: Billing). - Score Head: Outputs a calibrated numeric value (e.g.,
Confidence: 0.95). - Boolean/Noul Head: Outputs a true/false flag.
3. A Single Forward Pass
Because the heads are independent and non-autoregressive, the model runs a single forward pass through the network. It does not loop. It takes the input, processes it, and spits out all the typed decisions at once. This reduces latency from thousands of milliseconds down to ~30-50ms.
Watch Out For
Treating it as a Generator Do not use a decision model when you actually need to synthesize new text, such as drafting a response email or summarizing a document. Decision models do not generate tokens; they only classify and route. If you need text, you need a standard LLM.
The Quick Version
- Autoregressive models are too slow and error-prone for simple routing tasks because they generate text token by token.
- System 1 decision models use an encoder to read the input in a single pass.
- Typed heads sit on top of the encoder to output strict choices and scores directly.
- No text is generated, eliminating hallucinations and the need for JSON parsing.
- Latency is reduced to tens of milliseconds, making them ideal for front-line agent routers.
What to Read Next
To understand the models they replace, see Transformers.