Unlearning
The complex process of intentionally removing specific knowledge, facts, or behaviors from a trained model's weights without degrading general performance.
Think of It Like This
Like trying to perfectly remove just the salt from a fully baked cake without ruining the texture, flavor, or structural integrity of the dessert.
Machine unlearning is an active research area driven by privacy regulations (like the Right to Be Forgotten) and copyright concerns. Since an LLM's knowledge is deeply entangled across billions of parameters, simply deleting a fact is impossible. Techniques often involve gradient ascent on the forgotten data to actively destroy those pathways.