Learning Rate Scheduling
Changing the learning rate over the course of training on a fixed schedule, usually shrinking it, instead of holding one value for every step.
Step decay drops the rate by a fixed factor at set intervals, in sudden jumps. Cosine annealing follows a smooth curve down to near zero over the whole run instead, which is why loss curves under it tend to look steadier near the end of training.
Warmup-stable-decay is the shape the largest training runs favor: ramp up, hold a long flat plateau that can be extended without reshaping anything, then ease down at the end — useful specifically because it doesn't require committing to a fixed total step count up front.