Model Merging
Two fine-tunes that started from the same base can be combined by adding their weight changes, so you get one model with both skills and no retraining.
Why Does This Exist?
You have a math expert and a code expert. Both started life as the same base checkpoint, and now you want one assistant that handles both. You could mix the two datasets and pay for joint retraining. That means new data plumbing, a new training run, and a fresh round of evals. When the experts already exist, you'd rather combine what you have.
Model merging skips the retrain with arithmetic. You subtract the base from each expert to isolate what each fine-tune added, then add chosen fractions of those changes back onto the base. No gradients. No training data. The merge itself runs in seconds on a CPU, which is why small teams reach for it before they book more GPUs.
This page covers that combination step. How you build each expert lives in full fine-tuning and supervised fine-tuning. Cheap ways to produce experts you later merge live in LoRA and QLoRA. Here you start with finished experts and end with one model.
You need one idea to follow the math. Fine-tuning moves base weights , a plain list of numbers, to fine-tuned weights by training on task data. The shift is what merging reuses. If that feels shaky, read full fine-tuning first, then come back.
Think of It Like This
Two punch lists on one blueprint
You and a friend buy the same flat-pack bookshelf. You keep the original instruction sheet untouched. You each write a separate punch list of tweaks. Yours widens the top shelf for textbooks. Hers adds a cable hole for a lamp. To get both upgrades, you take a fresh copy of the original instructions and apply both punch lists. You don't rebuild the shelf twice.
Base weights are the original instructions. Each fine-tune change is one punch list: only the edits. The mixing weight says how much of each list you apply. Merging works because both lists edit the same numbered parts.
Where it stops: shelf tweaks rarely collide, but weight tweaks do. If both lists drill the same hole in different places, you can't apply both fully. That's interference, and that's what the rest of this page handles.
How It Actually Works
Merging only makes sense because the experts share coordinates. Same architecture, same starting checkpoint, same position for every number. You'll tune how much of each move you keep, and you'll test the blend per task. Merging never invents knowledge. If neither expert saw a skill, the merged model won't have it.
Task vectors isolate what each expert added
Write for the base weights, a flat list of numbers in a fixed order. Write and for two fine-tunes from that base. In plain words: is where you started, is where task A took you, is where task B took you.
Define the task vector , entry by entry. Read as tau, the change. is the math skill as pure movement. is the code skill as pure movement. Each has the same shape and order as the base, so addition lines up.
The merge rule adds chosen fractions back
Pick mixing weights and , plain numbers that say how much of each skill you keep. Read as lambda. The merged weights are:
In plain words: start from the base, then walk partway along each task move and stop. Setting both values to averages the two experts. Lower values keep less of each skill but cut interference. You'd tune them on held-out checks, the same way you'd tune any blend.
If you can afford two models at serve time, a deep ensemble keeps both experts intact and often scores higher. Merging buys you one model at the cost of some washout.
Why straight averaging can sag
Simple averaging joins points with a straight chord. In high dimensions that chord cuts inside the sphere where good weights live, so its length shrinks and behavior softens. Spherical interpolation, called SLERP, walks along the surface instead.
For two weight vectors and with angle between them, read as omega, and blend fraction between and , SLERP weights and by sines of that angle so the path keeps its radius. Use it when you blend whole weights from one base and care about preserving scale. Use task-vector scaling when you want a separate dial per skill.
How TIES calms interference
Two task vectors often push the same coordinate opposite ways. One says plus, the other says minus. Averaging keeps a watered-down remainder and both skills slip. That washout grows as you add a third or fourth expert.
TIES answers in three passes. Trim keeps only large-magnitude entries and drops noisy small shifts. Elect votes on the sign per coordinate and keeps the majority direction. Merge averages only entries that agree with that direction. You'll lose some detail, but the surviving direction stays clean. When tasks fight over most coordinates, that's a signal to merge fewer models or pay for joint training.
Worked example with two weights
Keep two numbers so you can check each step. Let base weights be . Let expert A reach and expert B reach .
Subtract to get task vectors:
Both coordinates disagree in sign. That's interference you can see.
Merge with and :
Half of is . Half of is . Their sum is . Add the base:
The diagram shows these same numbers. Simple averaging of and gives the same answer here, since and . The task-vector form earns its keep when you want unequal dials, or when TIES must drop a conflicting entry before averaging.
Code
Use this to check your grasp. It rebuilds the worked example with plain lists, so the prints must match the hand math.
def task_vector(base: list[float], finetuned: list[float]) -> list[float]: return [f - b for b, f in zip(base, finetuned)]
def merge( base: list[float], deltas: list[list[float]], lambdas: list[float]) -> list[float]: merged = base[:] for delta, lam in zip(deltas, lambdas): for i, d in enumerate(delta): merged[i] += lam * d return merged
w0: list[float] = [1.0, 2.0]wA: list[float] = [1.5, 1.75]wB: list[float] = [0.75, 2.5]tauA = task_vector(w0, wA)tauB = task_vector(w0, wB)merged = merge(w0, [tauA, tauB], [0.5, 0.5])print(tauA) # -> [0.5, -0.25]print(tauB) # -> [-0.25, 0.5]print(merged) # -> [1.125, 2.125]The three prints repeat the hand calculation: both task vectors, then the merged pair. Change a lambda to and you'll see the merged point slide toward the base. That slide is the trade you'd tune on held-out checks.
Watch Out For
Merging across different bases gives fluent garbage
The symptom is a model that talks smoothly but knows nothing. Evals collapse on every task at once, and the outputs drift off topic. You'll want to blame the merge weights, but the weights were never comparable.
The cause is ancestry. Different base checkpoints assign different meanings to the same coordinate, even when shapes match. Check the base checkpoint hash, architecture, and tokenizer before you merge. Merge only fine-tunes from one shared base. If shapes differ, stop. If the bases differ but shapes match by luck, still stop.
Full-strength blends wash out both skills
The symptom is a merge that scores below both parents on their own tasks. Code answers lose edge cases. Math answers skip steps. Everything reads a bit generic.
The cause is sign interference. Opposite pushes cancel, and the damage grows with more experts or with each left at . Lower the lambdas, test each task separately, and keep only large-magnitude entries before averaging. Merge fewer models. If both skills must stay at peak, keep an ensemble or pay for joint training instead of forcing the blend.
The Quick Version
- Model merging combines fine-tunes that share one base by adding scaled task vectors, with no training run.
- A task vector is fine-tuned weights minus base weights, so it isolates what one task added.
- The merge rule is base plus each task vector times its mixing weight, and 0.5 each equals a simple average.
- SLERP blends along the sphere to preserve scale, while TIES trims noise, votes on sign, and averages only agreeing entries.
- Do not merge models from different bases, and expect some washout: if both skills must stay peak, keep an ensemble or train jointly.