Apply LoRA Adapters

Fine-tuning a full 7-billion parameter model is expensive — you need multiple GPUs, days of training, and a heavy serving stack. If you've already trained a LoRA adapter on a small, focused dataset, you now face the next question: how do you actually put this adapter onto a base model so you can generate text with it? This lesson walks you through the exact process of applying a LoRA adapter to a base model, using the Hugging Face peft library. You'll learn the mental model, the step-by-step mechanics, and how to avoid the common pitfalls that trip up beginners — so you can ship your first adapted model with confidence.

The problem this lesson solves

You've trained a LoRA adapter and now you have a file named adapter_model.safetensors sitting in your output directory. The adapter alone is useless — it contains only the low-rank update matrices, not the full language model. To make predictions, you need to apply that adapter to the base model it was trained on. This process is called loading or merging the adapter. Without doing it correctly, you'll either get an error about mismatched keys or generate complete nonsense.

The core challenge is that a LoRA adapter is tightly coupled to its base model. The adapter's target_modules (the names of the layers that were modified) and the base model's architecture must match exactly. If you swap the base model, the adapter won't load, or worse, it will load silently but produce garbage outputs. This lesson gives you the exact steps to apply the adapter safely.

Core concept / mental model

Think of a LoRA adapter as a software patch for a large application. The base model is the original app — complete and functional on its own. The adapter is a small diff file that changes a few functions to behave differently. Applying the adapter means applying that patch, and you can do it in two ways:

• Attach the patch temporarily (load the adapter) — the base model is unchanged, and you can remove the patch at any time.
• Commit the patch permanently (merge the adapter) — you create a new version of the base model that has the patch baked in, so you no longer need the adapter file.

In technical terms, a LoRA adapter adds trainable low-rank matrices (called lora_A and lora_B) alongside the original weight matrices in the attention layers. During training, only these new matrices change; the original weights stay frozen. When you apply the adapter, you tell the base model to inject these matrices into the forward pass, effectively altering its behavior without modifying the original parameters.

The key mental model: base model + adapter = fine-tuned model, but only when the adapter is correctly loaded or merged.

The peft library abstracts this beautifully. You don't manually manipulate tensors; you use PeftModel.from_pretrained() to load the adapter, and merge_and_unload() to merge it into the base model.

How it works step by step

The process of applying a LoRA adapter follows a reliable sequence:

1. Load the base model exactly as it was when you trained the adapter. This means the same architecture, same tokenizer, and ideally the same version from Hugging Face Hub.
2. Load the adapter using PeftModel.from_pretrained(base_model, adapter_path). The adapter metadata tells peft which layers to modify.
3. Decide: load for inference or merge permanently. If you want to keep your base model intact, just use the PeftModel as is. If you want to deploy a single file (or integrate with other tools), call merge_and_unload() to combine the adapter into the base model's weights.
4. Verify by generating output with both the base model and the applied adapter — a quick sanity check to ensure the adapter is actually influencing the output.

The step-by-step cause and effect is simple: base model weights stay frozen, adapter matrices are added to the forward pass, and your generation changes accordingly.

Hands-on walkthrough

Let's walk through a complete example where we apply a trained LoRA adapter to a base model. We'll use a small model like facebook/opt-350m and a simple adapter from a hypothetical training run. In practice, you'll use the adapter path you saved from earlier training.

Step 1: Install the required libraries

pip install transformers peft accelerate safetensors torch

Step 2: Load the base model and tokenizer

from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_name = "facebook/opt-350m"
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    device_map="auto",  # automatically distribute to available GPUs
)

Step 3: Apply the LoRA adapter

from peft import PeftModel

# Path to your training output directory
adapter_path = "./lora-adapter"  # contains adapter_model.safetensors and adapter_config.json

model = PeftModel.from_pretrained(base_model, adapter_path)

That's it — your model now has the adapter attached.

Step 4: Generate text with the adapted model

prompt = "The best thing about Paris is"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda" if torch.cuda.is_available() else "cpu")

output = model.generate(
    **inputs,
    max_new_tokens=50,
    do_sample=True,
    temperature=0.7,
)

text = tokenizer.decode(output[0], skip_special_tokens=True)
print(text)

Expected output: (varies by training data) something like:

The best thing about Paris is the way the light hits the Seine at dusk, casting a warm glow over the cafes...

Step 5: Merge and unload the adapter (optional)

If you want a standalone model for deployment, merge the adapter into the base model and save the result.

merged_model = model.merge_and_unload()
merged_model.save_pretrained("./merged-model")
tokenizer.save_pretrained("./merged-model")

Now you have a standard transformers model that doesn't require peft at load time — perfect for tools that don't know about LoRA.

Pro tip: Always verify your adapter works by comparing generation output with and without the adapter. A good adapter should produce noticeably different (fine-tuned) text on your domain.

Compare options / when to choose what

There are two main ways to apply a LoRA adapter: load without merging and merge into the base model. Each has trade-offs.

Decision rule: If you are experimenting or need to serve multiple fine-tuned variants of the same base model, keep the adapter separate. If you are shipping a single model to production, merge it.

Troubleshooting & edge cases

Mismatched base model or adapter config

You'll get an error like KeyError: 'model.layers.0.self_attn.q_proj.lora_A.weight' if the base model doesn't have the same architecture as the adapter expects. Fix: load the exact base model name used during training.

Adapter loaded but output is unchanged

The adapter is not being applied — often because the model was loaded with device_map incorrectly or the adapter was not attached. Verify that model is a PeftModel (print model.__class__).

Memory errors when loading large models

If the base model is too large for your GPU, use load_in_8bit=True or load_in_4bit=True in from_pretrained to quantize it. But be careful: merging a LoRA adapter into a quantized model requires merge_and_unload(progressbar=True) and may fail; consider keeping the adapter separate.

Tokenizer mismatch

If your training data used a custom tokenizer, apply the same tokenizer when loading the base model. Otherwise, your generation will be garbled.

Loading multiple adapters

You can load multiple adapters into the same base model using model.load_adapter(adapter2_path, adapter_name="adapter2"). Then switch with set_adapter(adapter_name). Great for A/B testing.

What you learned & what's next

You now know how to apply a LoRA adapter to a base model — the core concept that adapters are additive patches, and the two main approaches: loading for flexible experimentation and merging for deployment. This skill directly enables you to take any fine-tuned LoRA adapter and turn it into a working model. In the next lesson, you'll learn how to evaluate your fine-tuned model — comparing its output against a baseline to measure quality improvements. This is essential for proving that your finetuning effort actually paid off.

Keep this mental model in mind: base model + adapter = fine-tuned model, but you must apply it correctly to see the benefit.