GRPO
A reinforcement learning technique for alignment that optimizes policy updates using grouped responses, improving sample efficiency without memory overhead.
Think of It Like This
Like a teacher grading a group project by comparing students against their peers in the same group rather than against the entire school district standard.
Group Relative Policy Optimization (GRPO) was designed to bypass the need for an external value model during reinforcement learning. By sampling multiple outputs for a single prompt and normalizing their rewards relative to the group, GRPO effectively estimates the baseline. This dramatically reduces the memory footprint required to align highly capable language models.