Load Pretrained Models from Hugging Face Hub

The problem is usually not the model itself — it's the setup. You've just finished preparing a dataset, and now you need a pretrained model to fine-tune. Downloading the right model from the Hugging Face Hub, getting the tokenizer to match, and ensuring everything works on your hardware can eat hours if you guess wrong. This lesson shows you the battle-tested way to load a pretrained model and tokenizer in a handful of lines — and how to avoid the silent bugs that come from mismatched checkpoints.

The problem this lesson solves

When you start an LLM fine-tuning project, you might spend days gathering data, cleaning it, and formatting prompts. But the moment you try to load a model into memory, you hit a wall. Maybe the model name is wrong, the tokenizer produces different token IDs than the model expects, or the model is too large for your GPU and crashes with an out-of-memory error.

The Hugging Face Hub hosts over a million models, but that abundance is also a trap. You can’t just download any model and expect it to work with your task. The model architecture must match the tokenizer, the model must be compatible with your library (Transformers, PEFT, etc.), and the checkpoint must be something your hardware can handle. This lesson teaches you how to load a pretrained model from Hugging Face Hub correctly, so you can move straight to fine-tuning instead of debugging imports.

Pro tip: The model card on the Hub is your first stop. It lists the exact model ID, the tokenizer to use, and the recommended hardware. Skim it before writing a single line of code.

Core concept / mental model

Think of the Hugging Face Hub as a giant library where every book is a pretrained model. Each model has a unique model ID like google-bert/bert-base-uncased or mistralai/Mistral-7B-Instruct-v0.3. When you call AutoModel.from_pretrained(model_id), you’re asking the library to fetch the right “book” from the shelf, download it, and load it into memory.

The key idea is the Auto classes. Instead of manually writing BertModel or GPT2Model, the AutoModel and AutoTokenizer classes inspect the checkpoint’s config to determine the architecture and load the correct class. This means you can switch between models by changing only the model ID — the rest of your code stays the same.

There’s a critical relationship between the model and the tokenizer. The model expects token IDs that its vocabulary understands. The tokenizer converts text to those IDs and back. If they don’t match — for example, you use a GPT-2 tokenizer with a BERT model — the model will produce garbage. That’s why you almost always load both from the same model ID.

How it works step by step

Loading a pretrained model from the Hugging Face Hub follows a simple, repeatable process:

1. Install the required libraries — transformers, torch, and optionally accelerate for large models.
2. Choose a model ID — pick a model that works for your task and fits your hardware. Smaller models like BERT are great for classification; large ones like Llama-3-8B are for generation.
3. Load the tokenizer — use AutoTokenizer.from_pretrained(model_id) to get the tokenizer that was trained with the model.
4. Load the model — call AutoModelForSequenceClassification.from_pretrained() (or the task-specific variant) with the same model ID.
5. Move the model to your device — .to("cuda") if you have a GPU, else CPU. For large models, use device_map="auto" to spread weights across GPU/CPU or disk.
6. Test the loading — feed a short sample text through the tokenizer and model to confirm everything works before starting training.

The cause-and-effect here is straightforward: correct model ID + matching tokenizer = functional model. Any mismatch will either raise an error (good) or silently produce wrong outputs (bad).

Hands-on walkthrough

1. Install dependencies

If you haven’t already, install the core libraries. Use a virtual environment to avoid conflicts.

pip install transformers torch accelerate

2. Load a BERT model for sequence classification

Let’s load a small BERT model fine-tuned for sentiment analysis. This is a common starting point for fine-tuning on your own data.

from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_id = "nlptown/bert-base-multilingual-uncased-sentiment"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

# Now test the model with a quick forward pass.
inputs = tokenizer("This tutorial is super helpful!", return_tensors="pt")
outputs = model(**inputs)

# The logits show the model’s raw predictions.
print(outputs.logits)

Expected output (the actual numbers will vary):

tensor([[ 0.1123, -0.0456,  1.2394,  2.0112,  0.8671]])

This output is a vector of 5 values, one per star rating (1 to 5). The highest value (index 3) suggests a 4-star review.

3. Load a decoder-only model for generation

Now let’s load a generative model like GPT-2. No task-specific head — just the base model and its tokenizer.

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "gpt2"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Add a padding token (GPT-2 has none by default; we'll fix this later).
tokenizer.pad_token = tokenizer.eos_token

# Generate a small completion.
inputs = tokenizer("The capital of France is", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=10)
print(tokenizer.decode(outputs[0]))

Expected output (approximate):

The capital of France is Paris and it is famous for its

4. Load a huge model with device mapping

For models larger than your GPU, use device_map and torch_dtype to reduce memory.

from transformers import AutoModelForCausalLM

model_id = "mistralai/Mistral-7B-Instruct-v0.3"
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",       # spread weights across GPU/CPU/disk
    torch_dtype="auto"       # use the checkpoint's data type (often bfloat16)
)

# The tokenizer is loaded the same way as before.

Pro tip: device_map="auto" is your friend. It automatically places layers on available GPUs, then CPU, then disk (with a warning). This lets you load models that would otherwise OOM.

Compare options / when to choose what

The Hugging Face Hub offers several ways to load models. Here’s when to use each:

In short, Auto classes are the default choice because they let you swap models without changing code. Switch to explicit classes only when you need to pin a specific architecture for reproducibility.

Troubleshooting & edge cases

Even with the right approach, you’ll hit issues. Here are the most common ones and how to fix them:

1. RustPanicException: Unable to load the vocabulary file

Cause: The model ID is incorrect or doesn’t have a tokenizer file (e.g., a raw checkpoint without tokenizer.json).

Fix: Double-check the model ID in the Hub URL. Search for a model with that exact name. If you’re using a custom checkpoint, make sure the repo contains a tokenizer.json or vocab.txt.

2. “The model class you are using is not compatible with the model you are loading”

Cause: You’re using AutoModelForSequenceClassification for a model that wasn’t trained with a classification head (like a raw BERT or a decoder-only model).

Fix: Switch to AutoModelForCausalLM for generative models, or AutoModel if you only need the base. Check the model card to see the recommended class.

3. Out-of-memory (CUDA OOM) error

Cause: The model is too large for your GPU’s VRAM.

Fix: Use device_map="auto" to offload layers to CPU or disk. Alternatively, load in torch.float16 (torch_dtype=torch.float16). For very large models, consider quantization with bitsandbytes (covered in later lessons).

4. Tokenizer produces different token IDs than the model expects

Cause: You loaded a tokenizer from a different model ID than the model itself.

Fix: Always use the same model_id for both AutoTokenizer and the model. If you must pair them, ensure the vocab sizes align — check tokenizer.vocab_size vs model.config.vocab_size.

5. Download is extremely slow or times out

Cause: Large model files (several GB) and a slow network.

Fix: Use the HF_HUB_ENABLE_HF_TRANSFER=1 environment variable (requires pip install hf_transfer) for faster downloads. Or use the mirror parameter if you’re in a region with restricted access.

What you learned & what's next

You now know how to load a pretrained model from Hugging Face Hub — the first real step in any fine-tuning project. You can:

• Use AutoTokenizer and AutoModelForX to load any model with just its ID.
• Handle device placement with device_map and memory-saving data types.
• Debug common loading errors related to mismatched tokenizers, wrong model classes, and out-of-memory errors.

This foundation prepares you to start customizing these models. In the next lesson, you’ll learn how to tokenize and pad your dataset to match the model’s input format — a crucial step before training. With the model and tokenizer correctly loaded, you’re ready to feed your data through it and watch it learn. Keep your model loading code handy; you’ll be reusing it every time you train or evaluate a model.