LLM Engineering & Generative AI · LLM Engineering & Generative AI

LLM Finetuning

This track covers the practical skills for adapting pretrained large language models to your own data. You'll learn data preparation, parameter-efficient methods like LoRA and QLoRA, training loops with Hugging Face Transformers, and evaluation strategies. Ideal for ML engineers and developers wanting to customize LLMs for specific tasks, it ends with deploying a fine-tuned model responsibly.

Start tutorial

Sponsored

Lesson outline

  1. 13
    Set Training Hyperparameters for LLMs

    Set training hyperparameters for LLMs in this LLM Finetuning tutorial — practical steps, common pitfalls, and what to study next.

  2. 14
    Monitor Training Loss & Overfitting

    Learn to monitor training loss and detect overfitting in LLM fine-tuning. This hands-on lesson covers loss curves, validation metrics, and practical troubleshooting.

  3. 15
    Evaluate a Baseline Pretrained Model

    Learn to evaluate a baseline pretrained model before fine-tuning, using practical steps for LLM Finetuning. Set up metrics, run the baseline, and interpret results to improve your fine-tuning strategy.

  4. 16
    Catastrophic Forgetting in LLMs

    Understand catastrophic forgetting in LLMs — why fine-tuned models lose prior knowledge, and how to detect and mitigate it. Hands-on troubleshooting, edge cases, and next steps included.

  5. 17
    Define a Prompt Template

    Learn how to define a prompt template for task data in this LLM Finetuning tutorial. Step 17 covers the core concept, hands-on implementation, common pitfalls, and next steps.

  6. 18
    Create an Instruction Dataset

    Learn how to create an instruction-style dataset for fine-tuning LLMs.

  7. 19
    Chat Formatting with Tokenizers

    Use tokenizers for chat formatting in LLM finetuning — step-by-step guide with hands-on exercise, troubleshooting, and next steps.

  8. 20
    Freeze Model Layers for Transfer Learning

    Learn how to freeze model layers for transfer learning in this LLM finetuning tutorial. Master the core concept, apply it hands-on, and prepare for the next lesson.

  9. 21
    Full Finetuning vs Feature Extraction

    Compare full finetuning vs feature extraction for LLMs: understand trade-offs in cost, performance, and data needs. Hands-on comparisons guide your choice.

  10. 22
    Downscale with Gradient Accumulation

    Downscale with gradient accumulation — LLM Finetuning.

  11. 23
    Mixed Precision GPU Training

    Learn how to use mixed precision training on GPU to speed up LLM fine-tuning while saving memory. Practical steps, troubleshooting, and next steps in this tutorial.

  12. 24
    LLM Truncation Basics

    Handle long sequences with truncation in LLM fine-tuning. Learn how to truncate tokenized inputs efficiently, avoid data loss, and keep training runs fast and stable.

    LLM Finetuning — step-by-step tutorials

    What you will find here

    This track walks through llm finetuning in order — each lesson is server-rendered HTML you can read without JavaScript. Follow the outline, then practice in the browser IDE when a lesson links to runnable code.

    Tutorials vs quizzes and code samples

    Tutorials teach in sequence. For quick checks use quizzes. For copy-paste snippets see code samples. For deeper reading browse articles.