Merge LoRA Weights

You've spent hours fine-tuning a model with LoRA, watching the loss curve drop, and finally you have an adapter that performs exactly how you want. But now comes the moment of truth: you can't just ship those adapter weights to production. Your deployment server won't know what to do with a separate adapter file, and inference will be painfully slow if you try to load a base model and an adapter on every request. This lesson shows you how to merge LoRA weights into the base model, turning your fine-tuned adapter into a standalone, deploy-ready model that runs like any other pretrained checkpoint.

The problem this lesson solves

When you fine-tune with LoRA, you don't save a full copy of the model. Instead, you save two things: the original pretrained weights (which stay frozen) and a small set of low-rank adapter weights. That's great for training efficiency, but it creates a headache for deployment:

• Your adapter is useless without the base model. If someone uses your model without loading the base weights, they get garbage output.
• The base model and adapter must stay in sync. If you upgrade the base model version, your adapter may break.
• Inference becomes slower and more complex. Every forward pass must apply the adapter computation on top of the base layers, which adds overhead.
• The adapter format isn't universal. Hugging Face's peft library saves adapters in its own format, which some inference servers don't understand.

Merging solves all of this. By mathematically combining the LoRA weights into the base model's weight matrices, you produce a single, standard model checkpoint that can be loaded, saved, and served just like any other LLM. No special libraries, no extra steps — just a model.

Core concept / mental model

Think of LoRA as post-it notes stuck onto a textbook. The textbook is the base model — its content (weights) stays the same. The post-it notes (the low-rank adapter) contain only the corrections or additions you made during fine-tuning. You can read the notes only if you have the textbook nearby.

Merging is the act of copying the post-it notes into the textbook itself. Once you've done that, you no longer need the notes. The textbook now contains the improved content directly.

Technically, LoRA decomposes the weight update into two low-rank matrices, A and B. For a given layer, the modified weight matrix is:

W' = W + (alpha / r) * B @ A

where W is the original weight matrix, r is the rank, and alpha is a scaling hyperparameter. Merging computes this addition for every affected layer and writes the result back as the new weight matrix of the base model.

🧠 Key mental model: Merging is a one-time arithmetic operation, not a training step. You're not learning anything new; you're baking the learned adapter permanently into the weights.

How it works step by step

Merging LoRA weights into the base model follows a straightforward, deterministic process:

1. Load the base model using the same model class you used during training (e.g., AutoModelForCausalLM).
2. Load the LoRA adapter on top of it using PeftModel.from_pretrained.
3. Trigger the merge with model.merge_and_unload().* This method combines the adapter weights with the base weights and then removes the adapter layers, leaving you with a standard, unmodified model object.
4. Save the merged model to disk using model.save_pretrained() — along with its tokenizer.
5. Reload and test the merged model to verify it produces the same outputs as the original (adapter + base) combination.

*The unload() part is crucial — it strips out the peft wrapping, so the model is no longer a PeftModel but a plain AutoModelForCausalLM.

Why not just use the adapter at inference?

You can, and for quick experiments that's fine. But as you move to production, a merged model wins on simplicity and speed. There's no adapter loading step, no custom inference handler, and the model is fully compatible with standard serving tools like vLLM and TGI.

Hands-on walkthrough

Let's do it in practice. You'll need transformers and peft installed. We'll use the classic facebook/opt-350m as the base and assume you have a trained adapter saved in ./my_adapter.

1. Merge the adapter into the base model

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# 1. Load the base model
base_model = AutoModelForCausalLM.from_pretrained(
    "facebook/opt-350m",
    torch_dtype="auto",
    device_map="auto"
)

# 2. Load the LoRA adapter
model = PeftModel.from_pretrained(base_model, "./my_adapter")

# 3. Merge and unload
merged_model = model.merge_and_unload()

# 4. Save the merged model and tokenizer
merged_model.save_pretrained("./merged_model")
tokenizer = AutoTokenizer.from_pretrained("facebook/opt-350m")
tokenizer.save_pretrained("./merged_model")

print("Merged model saved to ./merged_model")

Expected output:

Merged model saved to ./merged_model

2. Verify the merge worked

Now load the merged model and compare its output with the original (unmerged) adapter. They should be identical (or extremely close) on the same input.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load merged model
merged = AutoModelForCausalLM.from_pretrained("./merged_model")
merged_tokenizer = AutoTokenizer.from_pretrained("./merged_model")

# Load original (unmerged) for comparison
base = AutoModelForCausalLM.from_pretrained("facebook/opt-350m")
original = PeftModel.from_pretrained(base, "./my_adapter")

prompt = "The capital of France is"
inputs = merged_tokenizer(prompt, return_tensors="pt")

tokens_merged = merged.generate(**inputs, max_new_tokens=10)
tokens_original = original.generate(**inputs, max_new_tokens=10)

print("Merged:", merged_tokenizer.decode(tokens_merged[0], skip_special_tokens=True))
print("Original:", merged_tokenizer.decode(tokens_original[0], skip_special_tokens=True))

Expected output:

Merged: The capital of France is Paris.
Original: The capital of France is Paris.

If the outputs match, your merge is correct. If they differ significantly, something went wrong — check the troubleshooting section.

3. Push to Hugging Face Hub (optional)

Merged models are perfect for sharing. You can upload it to the Hub and let others use it without touching peft:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("./merged_model")
tokenizer = AutoTokenizer.from_pretrained("./merged_model")

model.push_to_hub("your-username/your-model")
tokenizer.push_to_hub("your-username/your-model")

Compare options / when to choose what

Should you always merge? Not necessarily. Here’s a quick comparison:

When merging is a no-brainer

• Deploying a single fine-tuned model. If you don't need to swap adapters at runtime, merge.
• Quantizing for edge devices. Merged models compress better with tools like GPTQ.
• Removing dependency on peft. Your serving stack becomes simpler.

When to keep the adapter separate

• Serving many adapters for many tasks using a single base model (e.g., with peft's PiSSA or a multi-LoRA inference server).
• You still want to fine-tune further. Keep the adapter so you can update it without touching the base.

Troubleshooting & edge cases

1. Outputs diverge after merge

If the merged model's outputs don't match the original adapter's outputs, check:

• alpha and r mismatch: Ensure you used the same r, alpha, and target_modules during training and merging. If those differ, the merge will compute the wrong sum.
• Base model mismatch: The adapter was trained on a different base version (e.g., facebook/opt-350m vs a cached custom version). Always load the exact same base model.
• Quantization mismatch: If the adapter was trained on a bitsandbytes 4-bit/8-bit base, you must merge with the same quantization settings.

2. merge_and_unload() fails with AttributeError

This usually means the model isn't a PeftModel. Make sure you created it with PeftModel.from_pretrained first.

3. Memory errors on large models

The merge is done in memory, so for a 70B model you need enough GPU or CPU RAM to hold both the base and merged weights momentarily. Use low_cpu_mem_usage=True and device_map="auto" to minimize peak memory.

4. Adapter contains tied embeddings or LM head

Some LoRA configs target the embedding or language modeling head. The merge still works, but ensure you load the full base model (including the head) before merging.

5. Half-precision vs full precision

If your base model was loaded in float16 and your adapter in float32, you may get precision mismatches. Maintain a consistent torch_dtype throughout.

What you learned & what's next

You now know how to merge LoRA weights into the base model: you can load a base model, apply a trained adapter, and produce a standalone checkpoint that works without peft and gives identical outputs. You also learned when to merge versus when to keep the adapter separate, and how to troubleshoot common issues like mismatched hyperparameters or memory limits.

In the next lesson, we'll explore evaluating your fine-tuned model — how to measure quality on a held-out test set, compare against the base, and decide if your adapter is actually ready for production. Merging is the last step before deployment, but evaluation ensures you only merge something worth shipping.