Choose a Model for Use Case

You've cleaned your data and know exactly which task your LLM needs to nail. But now you're staring at a wall of model cards — Llama, Mistral, Qwen, GPT-4o — each with different sizes, licenses, and quirks. Picking the wrong one can cost you weeks of training or a production model that's too slow and too expensive. This lesson gives you a repeatable framework to choose a model for your use case, so you start fine-tuning with confidence instead of guessing.

The problem this lesson solves

Choosing a model feels overwhelming because the LLM landscape changes weekly. You have open-weights giants like Llama 3.1 405B from Meta, smaller efficient models like Qwen 2.5 7B, and domain-specific options like BioMistral. Add in the trade-off between base models (pretrained on raw text) and instruct models (tuned to follow instructions), and you're stuck with analysis paralysis.

Most developers pick a model because it's popular or has good benchmarks. That's a trap. Benchmarks evaluate general knowledge, not your specific task. A model that scores 90% on MMLU can still fail at extracting invoice fields from Korean receipts. The real problem is threefold:

1. You overestimate your compute budget. A 70B model needs multi-GPU training and expensive inference.
2. You ignore your data's language and format. Models are trained mostly on English and common web text — your domain might be finance, medical, or code-heavy.
3. You conflate base and instruct models. Why fine-tune a base model when you need chat behavior? Or vice versa?

Asking "Which model is best?" is the wrong question. The right question: What are my constraints (compute, data, latency, licensing) and which model type fits them? This lesson gives you a structured way to answer that.

Core concept / mental model

Think of model selection like choosing a sports car vs. a delivery van. A sports car (huge 70B model) wins races but is impractical for hauling packages. A van (7B model) handles the daily route with far less fuel. You must know the job before you pick the vehicle.

The classic mental model is the capability vs. efficiency spectrum:

• Small models (1B–7B): Fast, cheap, run on a single GPU, handle narrow, well-defined tasks (classification, extraction, simple generation).
• Medium models (7B–13B): Balanced for general assistant tasks, tool calling, and coding when you have some GPU headroom.
• Large models (30B–70B+): Maximum reasoning and knowledge, but need heavy hardware for fine-tuning (e.g., 8×A100 80GB) and high-cost inference.

Another mental model is the base vs. instruct vs. chat divide:

• Base models (e.g., Llama-3.1-8B) produce raw text continuations. Fine-tune them for masked-language modeling or base-style generation.
• Instruct/Chat models (e.g., Llama-3.1-8B-Instruct) are pre-tuned to respond to instructions and hold conversations. They already know chat format, so fine-tuning is about injecting your task-specific behavior.

Finally, understand parameter count ≠ quality after fine-tuning. A small model trained on 50,000 high-quality examples can outperform a larger frozen model on a narrow task. The model is the canvas — your data paints the picture.

How it works step by step

Model selection is a deliberate sequence. Follow these five steps to make a defensible choice.

Step 1: Define your task type and output format

Write down what you need the model to do. Is it classification (spam vs. not), extraction (entity or field), generation (chat, summarization, code), or retrieval (embeddings)? The output format matters — if you must output JSON, you want a model that handles structured output well.

Step 2: Inventory your compute and latency budget

Count how many GPUs you have for fine-tuning and what inference latency your product tolerates. A 70B model needs ~140GB VRAM just for weights in FP16, plus optimizer states for training. If you only have one 24GB GPU, your ceiling is roughly a 7B model with QLoRA.

Step 3: Choose base vs. instruct/chat

If your task involves following instructions or dialogue, pick an instruct model. If you're doing text generation where you control the prompt entirely (e.g., creating product descriptions from structured fields), base models are fine and cheaper to run. Rule of thumb: always start with instruct unless you have a reason not to — instruct models give you guardrails and better zero-shot behavior.

Step 4: Pick a model family based on your domain

Different families shine at different tasks:

• Llama (Meta): Best general-purpose balance, strong reasoning, huge ecosystem, permissive license (for some sizes).
• Mistral / Mixtral: Excellent for code and efficiency, French provenance, popular for chat.
• Qwen (Alibaba): Strong multilingual support (especially Chinese), good for multilingual or tool-use tasks.
• Phi (Microsoft): Small (2.7B, 3.8B) but surprisingly strong for reasoning; perfect for edge devices.
• Domain-specific (BioMistral, FinGPT): Pre-tuned on vertical data, but check quality and update frequency.

Step 5: Validate with a mini benchmark on your data

Don't trust leaderboards alone. Take 100–200 samples from your training set, run zero-shot or few-shot prompts through 2–3 candidate models, and score them. This takes an hour but saves you weeks of wasted fine-tuning.

Hands-on walkthrough

Let's put the steps into practice. We'll evaluate two candidate models — microsoft/phi-3-mini-4k-instruct (3.8B) and meta-llama/Llama-3.1-8B-Instruct — on a sentiment classification task to decide which to fine-tune.

Example 1: Compare zero-shot performance

Use the Hugging Face transformers library to run both models on a few labeled examples.

from transformers import pipeline

def evaluate(model_name, sample_texts, labels):
    pipe = pipeline("text-classification", model=model_name, device=0)
    correct = 0
    for text, true_label in zip(sample_texts, labels):
        result = pipe(text)[0]["label"]
        # Normalize label format
        pred = "POSITIVE" if "POSITIVE" in result.upper() else "NEGATIVE"
        correct += (pred == true_label)
    return correct / len(sample_texts)

samples = [
    "This product exceeded my expectations!",
    "Terrible service, will never buy again.",
    "It's okay, nothing special.",
]
labels = ["POSITIVE", "NEGATIVE", "NEGATIVE"]

for model in ["microsoft/phi-3-mini-4k-instruct", "meta-llama/Llama-3.1-8B-Instruct"]:
    acc = evaluate(model, samples, labels)
    print(f"{model}: zero-shot accuracy = {acc:.2f}")

Expected output (example):

microsoft/phi-3-mini-4k-instruct: zero-shot accuracy = 0.67
meta-llama/Llama-3.1-8B-Instruct: zero-shot accuracy = 1.00

Llama scores better, but that's only 3 samples. In practice, you'd scale to at least 100.

Example 2: Estimate VRAM for fine-tuning

The second decision factor is hardware. Use this quick calculation for LoRA fine-tuning:

def estimate_vram(model_size_b: float, batch_size: int = 1, quantization: str = "none") -> float:
    """Rough VRAM estimate in GB for LoRA fine-tuning."""
    param_bytes = {"none": 4, "lora": 4, "qlora": 1}[quantization]
    # LoRA adds epsilon; base weights dominate
    weights_gb = model_size_b * param_bytes
    # Optimizer + gradients for LoRA are tiny, but add buffer
    overhead_gb = 0.5 * batch_size
    return weights_gb + overhead_gb

print(f"Phi-3 (3.8B) full precision: {estimate_vram(3.8):.1f} GB")
print(f"Llama-8B full precision: {estimate_vram(8):.1f} GB")
print(f"Llama-8B QLoRA: {estimate_vram(8, quantization='qlora'):.1f} GB")

Output:

Phi-3 (3.8B) full precision: 15.2 GB
Llama-8B full precision: 32.0 GB
Llama-8B QLoRA: 8.0 GB

On a single 24GB GPU, both fit, but Phi leaves room for longer sequences.

Example 3: Load and test generation speed

Inference speed matters. Test tokens per second:

import time
from transformers import pipeline

pipe = pipeline("text-generation", model="microsoft/phi-3-mini-4k-instruct", device=0)
prompt = "Write a product description for a wireless mouse."
start = time.time()
output = pipe(prompt, max_new_tokens=100)[0]["generated_text"]
latency = time.time() - start
print(f"Latency: {latency:.2f}s for 100 tokens")
print(f"Tokens/sec: {100 / latency:.1f}")

Expected: 50–100 tokens/sec on a V100. A 70B model would manage only ~5–10 tokens/sec.

Compare options / when to choose what

Below is a practical comparison to guide your choice after considering your constraints.

When to prioritize small models: You need low latency (<500ms), deploy on-device, have a narrow task, or have limited GPU budget. Small models fine-tuned on high-quality data often beat larger frozen ones on that specific task.

When to prioritize large models: Your task requires deep reasoning, complex instruction following, or broad knowledge (e.g., legal analysis). The cost is justified when a 2% accuracy improvement translates to significant business value.

When to consider a domain-specific model: You're in a niche like biomedicine or legal, and you have less training data. But always verify — a general model might already handle your domain well, and fine-tuning it could be easier.

Troubleshooting & edge cases

Model not following instruction format

If after fine-tuning your instruct model ignores system prompts or output format, you likely trained with inconsistent chat templates. Solution: Use tokenizer.apply_chat_template() to apply the exact template the model expects. Don't create your own separator tokens.

Out-of-memory errors during training

You picked a 13B model but only have one 24GB GPU. With QLoRA you can still proceed, but reduce batch size, use gradient accumulation, and enable gradient checkpointing. If it still OOMs, drop to a 7B.

Zero-shot performance is poor

Your chosen model (say a base model) can't handle your instruction-style task. Fix: Switch to an instruct variant. Base models need explicit prompting, and fine-tuning into instruct-style behavior requires far more data.

Overfitting on tiny datasets

The model memorizes training examples but fails on new data. This is a data problem, not a model problem. Ensure you have at least a few thousand examples, apply augmentation, and use strong regularization (weight decay, early stopping).

Benchmark scores mislead

A model with 90% MMLU fails your specific task. Cause: MMLU tests general knowledge, not domain-specific patterns. Always evaluate on your own test set — that's the only metric that matters for your use case.

What you learned & what's next

You can now explain the core idea behind choosing a model for your use case: matching capability and efficiency to your task, data, hardware, and budget. You completed a practical exercise comparing two models using zero-shot accuracy and VRAM estimates. You know how to apply the base vs. instruct distinction and use a comparison table to make a decision.

Key takeaways to remember:

• Always define your task type and constraints before browsing model cards.
• Instruct models are your default unless you need base behavior.
• Use your domain and data to narrow the model family.
• Validate candidates on your own labeled samples — benchmarks lie.
• Compute VRAM early to avoid picking an impossible model.

Next step: Once you've chosen a model, the next lesson in this track will show you how to prepare your dataset for fine-tuning. You'll learn how to format prompts, split training/validation sets, and create the right chat templates so your chosen model actually learns from your data. That's where the real magic begins.