Proximal Policy Optimization
A policy-gradient method that clips its objective so a single update can't push the policy too far from where it started, making it safe to reuse one batch for several steps.
The probability ratio between the new and old policy measures how far an update has already moved; the clipped objective stops rewarding further movement once that ratio leaves a narrow band around 1. This removes the incentive to overshoot, which makes reusing one batch of collected experience for several gradient steps safe instead of destabilizing.
That safe reuse of expensive rollouts is why PPO became the default optimizer behind reinforcement learning from human feedback.