Skip to content
AI360XpertBeta

Model Merging

Two fine-tunes that started from the same base can be combined by adding their weight changes, so you get one model with both skills and no retraining.

Two fine-tunes that share one base combine by adding scaled task vectors, giving one merged model with no training
Two fine-tunes that share one base combine by adding scaled task vectors, giving one merged model with no training

Why Does This Exist?

You have a math expert and a code expert. Both started life as the same base checkpoint, and now you want one assistant that handles both. You could mix the two datasets and pay for joint retraining. That means new data plumbing, a new training run, and a fresh round of evals. When the experts already exist, you'd rather combine what you have.

Model merging skips the retrain with arithmetic. You subtract the base from each expert to isolate what each fine-tune added, then add chosen fractions of those changes back onto the base. No gradients. No training data. The merge itself runs in seconds on a CPU, which is why small teams reach for it before they book more GPUs.

This page covers that combination step. How you build each expert lives in full fine-tuning and supervised fine-tuning. Cheap ways to produce experts you later merge live in LoRA and QLoRA. Here you start with finished experts and end with one model.

You need one idea to follow the math. Fine-tuning moves base weights W0W_0, a plain list of numbers, to fine-tuned weights WW by training on task data. The shift W−W0W - W_0 is what merging reuses. If that feels shaky, read full fine-tuning first, then come back.

Think of It Like This

Two punch lists on one blueprint

You and a friend buy the same flat-pack bookshelf. You keep the original instruction sheet untouched. You each write a separate punch list of tweaks. Yours widens the top shelf for textbooks. Hers adds a cable hole for a lamp. To get both upgrades, you take a fresh copy of the original instructions and apply both punch lists. You don't rebuild the shelf twice.

Base weights are the original instructions. Each fine-tune change is one punch list: only the edits. The mixing weight says how much of each list you apply. Merging works because both lists edit the same numbered parts.

Where it stops: shelf tweaks rarely collide, but weight tweaks do. If both lists drill the same hole in different places, you can't apply both fully. That's interference, and that's what the rest of this page handles.

How It Actually Works

Merging only makes sense because the experts share coordinates. Same architecture, same starting checkpoint, same position for every number. You'll tune how much of each move you keep, and you'll test the blend per task. Merging never invents knowledge. If neither expert saw a skill, the merged model won't have it.

Task vectors isolate what each expert added

Write W0W_0 for the base weights, a flat list of numbers in a fixed order. Write WAW_A and WBW_B for two fine-tunes from that base. In plain words: W0W_0 is where you started, WAW_A is where task A took you, WBW_B is where task B took you.

Define the task vector τA=WA−W0\tau_A = W_A - W_0, entry by entry. Read τ\tau as tau, the change. τA\tau_A is the math skill as pure movement. τB=WB−W0\tau_B = W_B - W_0 is the code skill as pure movement. Each has the same shape and order as the base, so addition lines up.

The merge rule adds chosen fractions back

Pick mixing weights λA\lambda_A and λB\lambda_B, plain numbers that say how much of each skill you keep. Read λ\lambda as lambda. The merged weights are:

Wmerged=W0+λAτA+λBτBW_{merged} = W_0 + \lambda_A \tau_A + \lambda_B \tau_B

In plain words: start from the base, then walk partway along each task move and stop. Setting both λ\lambda values to 0.50.5 averages the two experts. Lower values keep less of each skill but cut interference. You'd tune them on held-out checks, the same way you'd tune any blend.

If you can afford two models at serve time, a deep ensemble keeps both experts intact and often scores higher. Merging buys you one model at the cost of some washout.

Why straight averaging can sag

Simple averaging joins points with a straight chord. In high dimensions that chord cuts inside the sphere where good weights live, so its length shrinks and behavior softens. Spherical interpolation, called SLERP, walks along the surface instead.

For two weight vectors aa and bb with angle Ω\Omega between them, read Ω\Omega as omega, and blend fraction tt between 00 and 11, SLERP weights aa and bb by sines of that angle so the path keeps its radius. Use it when you blend whole weights from one base and care about preserving scale. Use task-vector scaling when you want a separate dial per skill.

How TIES calms interference

Two task vectors often push the same coordinate opposite ways. One says plus, the other says minus. Averaging keeps a watered-down remainder and both skills slip. That washout grows as you add a third or fourth expert.

TIES answers in three passes. Trim keeps only large-magnitude entries and drops noisy small shifts. Elect votes on the sign per coordinate and keeps the majority direction. Merge averages only entries that agree with that direction. You'll lose some detail, but the surviving direction stays clean. When tasks fight over most coordinates, that's a signal to merge fewer models or pay for joint training.

Worked example with two weights

Keep two numbers so you can check each step. Let base weights be W0=[1.0,2.0]W_0 = [1.0, 2.0]. Let expert A reach WA=[1.5,1.75]W_A = [1.5, 1.75] and expert B reach WB=[0.75,2.5]W_B = [0.75, 2.5].

Subtract to get task vectors:

τA=[1.5−1.0,1.75−2.0]=[0.5,−0.25]\tau_A = [1.5 - 1.0, 1.75 - 2.0] = [0.5, -0.25] τB=[0.75−1.0,2.5−2.0]=[−0.25,0.5]\tau_B = [0.75 - 1.0, 2.5 - 2.0] = [-0.25, 0.5]

Both coordinates disagree in sign. That's interference you can see.

Merge with λA=0.5\lambda_A = 0.5 and λB=0.5\lambda_B = 0.5:

Wmerged=[1.0,2.0]+0.5[0.5,−0.25]+0.5[−0.25,0.5]W_{merged} = [1.0, 2.0] + 0.5[0.5, -0.25] + 0.5[-0.25, 0.5]

Half of τA\tau_A is [0.25,−0.125][0.25, -0.125]. Half of τB\tau_B is [−0.125,0.25][-0.125, 0.25]. Their sum is [0.125,0.125][0.125, 0.125]. Add the base:

Wmerged=[1.125,2.125]W_{merged} = [1.125, 2.125]

The diagram shows these same numbers. Simple averaging of WAW_A and WBW_B gives the same answer here, since (1.5+0.75)/2=1.125(1.5 + 0.75)/2 = 1.125 and (1.75+2.5)/2=2.125(1.75 + 2.5)/2 = 2.125. The task-vector form earns its keep when you want unequal dials, or when TIES must drop a conflicting entry before averaging.

Code

Use this to check your grasp. It rebuilds the worked example with plain lists, so the prints must match the hand math.

def task_vector(base: list[float], finetuned: list[float]) -> list[float]:    return [f - b for b, f in zip(base, finetuned)]

def merge(    base: list[float], deltas: list[list[float]], lambdas: list[float]) -> list[float]:    merged = base[:]    for delta, lam in zip(deltas, lambdas):        for i, d in enumerate(delta):            merged[i] += lam * d    return merged

w0: list[float] = [1.0, 2.0]wA: list[float] = [1.5, 1.75]wB: list[float] = [0.75, 2.5]tauA = task_vector(w0, wA)tauB = task_vector(w0, wB)merged = merge(w0, [tauA, tauB], [0.5, 0.5])print(tauA)  # -> [0.5, -0.25]print(tauB)  # -> [-0.25, 0.5]print(merged)  # -> [1.125, 2.125]

The three prints repeat the hand calculation: both task vectors, then the merged pair. Change a lambda to 0.30.3 and you'll see the merged point slide toward the base. That slide is the trade you'd tune on held-out checks.

Watch Out For

Merging across different bases gives fluent garbage

The symptom is a model that talks smoothly but knows nothing. Evals collapse on every task at once, and the outputs drift off topic. You'll want to blame the merge weights, but the weights were never comparable.

The cause is ancestry. Different base checkpoints assign different meanings to the same coordinate, even when shapes match. Check the base checkpoint hash, architecture, and tokenizer before you merge. Merge only fine-tunes from one shared base. If shapes differ, stop. If the bases differ but shapes match by luck, still stop.

Full-strength blends wash out both skills

The symptom is a merge that scores below both parents on their own tasks. Code answers lose edge cases. Math answers skip steps. Everything reads a bit generic.

The cause is sign interference. Opposite pushes cancel, and the damage grows with more experts or with each λ\lambda left at 1.01.0. Lower the lambdas, test each task separately, and keep only large-magnitude entries before averaging. Merge fewer models. If both skills must stay at peak, keep an ensemble or pay for joint training instead of forcing the blend.

The Quick Version

  • Model merging combines fine-tunes that share one base by adding scaled task vectors, with no training run.
  • A task vector is fine-tuned weights minus base weights, so it isolates what one task added.
  • The merge rule is base plus each task vector times its mixing weight, and 0.5 each equals a simple average.
  • SLERP blends along the sphere to preserve scale, while TIES trims noise, votes on sign, and averages only agreeing entries.
  • Do not merge models from different bases, and expect some washout: if both skills must stay peak, keep an ensemble or train jointly.