Skip to content
AI360XpertBeta
Glossary
Definition

Bayesian Optimization

Choosing where to evaluate an expensive function next by fitting a surrogate model to past evaluations and ranking candidates with an acquisition function.

Think of It Like This

Like drilling exploratory oil wells by using past geological strikes to choose the next spot that balances high promise with uncharted territory.

A Gaussian process is the usual surrogate: it predicts both a value and an uncertainty at any point, tight near past evaluations and wide everywhere else. Expected improvement, the most common acquisition function, ranks a candidate high when the surrogate predicts a good result, high uncertainty, or both, balancing exploring the unknown against exploiting what already looks promising.

The overhead only pays off when a real evaluation is expensive, like a full training run; for anything fast to evaluate, plain random search usually wins on wall-clock time.