Concise, plain-English definitions of the AI and ML terms used across the site. Filter by tag to narrow the glossary.
The share of predictions a model gets right, usually written as a percentage. It's the simplest way to summarize how well a classifier is doing.
Analogy: It's a report-card percentage — 90% accuracy means 9 out of 10 calls were correct.
Related: loss, softmax
The little non-linear function each neuron applies to its input, letting a network learn curved, complex patterns instead of just straight lines. Without it, stacking layers would be pointless.
Analogy: The bend in the wire — without a kink, ten straight segments are still just one straight line.
Related: neural-network, softmax, gradient
Read the full conceptA mechanism that lets a model weigh which parts of the input matter most for each piece it's processing. Instead of treating every word equally, it focuses on the ones that are actually relevant.
Analogy: Like reading with a highlighter — you weight the important words and skim the rest.
Related: softmax, embedding
Read the full conceptThe number of training examples a model looks at before it updates its weights once. Bigger batches give steadier updates but need more memory; smaller batches update more often and add helpful noise.
Analogy: It's how many flashcards you review before pausing to update your notes.
Related: epoch, gradient, loss
The maximum amount of text — measured in tokens — a language model can consider at once, covering both your prompt and its reply. Go past it and the earliest tokens fall out of view.
Analogy: It's the model's desk space — only so much paper fits on top at one time.
Related: token, prompt, attention
Read the full conceptA way to test a model by splitting the data into several folds, training on most and testing on the rest, then rotating so every fold gets a turn as the test set. You get a sturdier estimate than a single split.
Analogy: Grading with several different pop quizzes instead of betting everything on one big exam.
Related: overfitting, underfitting, accuracy
A training trick that randomly switches off a fraction of a network's neurons on each step, forcing it not to lean on any single one. It's one of the simplest ways to fight overfitting.
Analogy: Cross-training a team so no single person is a point of failure.
Related: overfitting, neural-network, activation-function
Read the full conceptA way of representing words, images, or other items as lists of numbers (vectors) so a model can work with them. Items with similar meaning end up with similar vectors.
Analogy: It's a map where things with related meanings sit close together.
Related: attention, softmax, vector-database
One complete pass of the model over the entire training dataset. Models usually train for many epochs, seeing the data again and again to keep improving.
Analogy: One epoch is a single full read-through of the textbook — you normally need several before it sticks.
Related: batch-size, loss, gradient
A single number that balances precision and recall by taking their harmonic mean. It's high only when both are high, so it rewards models that are accurate and thorough at once.
Analogy: A combined grade that drops if you're weak in either subject.
Related: precision, recall, accuracy
Taking an already-trained model and training it a little more on your own examples so it specializes in your task, tone, or domain. You adjust an existing model rather than start from scratch.
Analogy: Hiring an experienced generalist and giving them a week of on-the-job training.
Related: transfer-learning, learning-rate, epoch
Read the full conceptA set of numbers that tells you which direction to nudge each of a model's weights to reduce the loss, and how strongly. It's the compass that points training downhill toward better predictions.
Analogy: Like feeling the slope under your feet in fog — it tells you which way is downhill and how steep.
Related: loss, batch-size, epoch
Read the full conceptWhen a language model states something false as if it were fact — a confident, fluent answer that simply isn't true. It happens because the model predicts plausible text, not verified truth.
Analogy: Like a smooth talker who would rather sound sure than admit they don't know.
Related: prompt, inference, fine-tuning
Read the full conceptA setting you choose before training that shapes how the model learns — like the learning rate, batch size, or number of layers. Unlike weights, the model never learns these; you pick them.
Analogy: The oven settings you dial in before baking, not the cake itself.
Related: learning-rate, batch-size, epoch
Read the full conceptRunning a trained model to get a prediction — the 'using it' phase, as opposed to training. Every time you send a prompt and get an answer back, that's one inference.
Analogy: Training is studying for the exam; inference is sitting it.
Related: token, temperature, prompt
Read the full conceptA setting that controls how big a step the model takes each time it updates its weights during training. Too big and it overshoots the target; too small and training crawls.
Analogy: Your step size walking downhill in fog — big strides are fast but risk flying past the bottom.
Related: gradient, loss, hyperparameter, batch-size
Read the full conceptA single number that measures how wrong a model's predictions are on the data it's shown. Training works by nudging the model to make this number as small as possible.
Analogy: Think of it as a golf score — the lower it goes, the better the model is playing.
Related: gradient, accuracy, epoch
Read the full conceptA model built from layers of simple connected units ('neurons') that transform an input into an output, learning the connection strengths from data. It's the workhorse behind modern AI.
Analogy: A relay team of simple runners — no single one is clever, but the handoffs add up to something powerful.
Related: activation-function, gradient, loss
Read the full conceptWhen a model memorizes its training data — noise and all — instead of learning the general pattern, so it aces training but flops on new data. The classic sign is great training scores and poor test scores.
Analogy: A student who memorizes the practice answers but can't handle a reworded question.
Related: underfitting, dropout, cross-validation
Read the full conceptOf all the times a model said 'yes,' the share that were actually right. High precision means few false alarms.
Analogy: A smoke detector with high precision rarely goes off just for burnt toast.
Related: recall, f1-score, accuracy
The text you feed a language model to steer its response — your question, your instructions, and any examples or context you include. It's the entire input the model reads before it writes a word.
Analogy: The prompt is the brief you hand a contractor: vague in, vague out.
Related: token, temperature, context-window, hallucination
Of all the real 'yes' cases out there, the share the model actually caught. High recall means few misses.
Analogy: A net with high recall catches almost every fish — even if it scoops up some seaweed too.
Related: precision, f1-score, accuracy
A function that turns a list of raw scores into probabilities that are all positive and add up to 1. It's how a model expresses how likely each option is.
Analogy: It turns raw scores into a pie chart — every slice is positive and they all add up to 100%.
Related: temperature, attention, accuracy
A setting that controls how random or predictable a language model's output is. Turn it low and the model plays safe, picking the most likely next word; turn it high and it takes more risks and sounds more creative.
Analogy: It's a creativity dial — low for focused answers, high for wilder ones.
Related: top-k, top-p, softmax
The small chunk of text a language model actually reads and predicts — often a whole word, a piece of a word, or a punctuation mark. Models measure and price everything in tokens, not characters.
Analogy: Tokens are the LEGO bricks of text — models build and bill in bricks, not whole sentences.
Related: embedding, context-window, inference
Read the full conceptA decoding rule that limits a language model to its K most likely next words, then samples from just that shortlist. It trims off the long tail of unlikely words so the output stays coherent.
Analogy: Order only from the menu's K bestsellers instead of the whole kitchen.
Related: top-p, temperature, softmax
A decoding rule that keeps the smallest set of next-word candidates whose probabilities add up to P, then samples from that set. The shortlist grows or shrinks depending on how confident the model is.
Analogy: Keep the likeliest words until they cover 90% of the odds, then stop.
Related: top-k, temperature, softmax
Reusing a model trained on one big task as the starting point for a different, usually smaller task. The features it already learned give the new task a huge head start.
Analogy: A guitarist picking up a mandolin — most of the musical skill carries straight over.
Related: fine-tuning, embedding, neural-network
Read the full conceptWhen a model is too simple to capture the real pattern in the data, so it does poorly even on the examples it trained on. Both training and test performance stay stuck.
Analogy: Drawing a straight line through data that clearly curves.
Related: overfitting, loss, accuracy
Read the full conceptA database built to store embeddings and find the ones most similar to a query, fast. It's what lets AI apps search by meaning instead of by exact keyword.
Analogy: A librarian who files books by what they're about, not by how their title is spelled.
Related: embedding, token, prompt
Read the full concept