Data Engineering · Data Engineering

Databricks

This track covers Databricks' unified analytics platform: Lakehouse architecture, cluster management, notebooks, and Delta Lake. You'll learn to build and run ETL pipelines, optimize Spark jobs, and enable data quality. It's for data engineers and analysts who want to operationalize data workloads. You'll leave able to design multi-stage ELT workflows and govern data on the Databricks platform.

Start tutorial

Sponsored

Lesson outline

  1. 37
    Reduce Cluster Startup with Pools

    Learn to use cluster pools to reduce startup time on Databricks. This hands-on tutorial explains how pools keep idle instances warm, cutting startup delays. Includes step-by-step setup, cost considerations, and troubleshooting tips.

  2. 38
    Databricks Autoscaling

    Configure autoscaling for dynamic workloads in Databricks. Learn how to set up and manage autoscaling clusters to handle variable workloads efficiently.

  3. 39
    Leverage Photon Engine

    Learn how Databricks' Photon engine accelerates SQL and DataFrame queries on Delta Lake. Understand when to enable it and how it improves performance.

  4. 40
    Inspect Spark UI for Tuning

    Learn how to inspect the Spark UI to tune and optimize your Databricks jobs. This hands-on tutorial covers key metrics, identifying bottlenecks, and practical next steps.

  5. 41
    Identify Shuffle Bottlenecks

    Identify shuffle bottlenecks in Spark jobs — Databricks.

  6. 42
    Tune Partition Counts

    Learn to tune partition counts for better performance in Databricks. This tutorial covers why partitions matter, how to choose the right count, hands-on examples, troubleshooting, and what to study next.

  7. 43
    Caching DataFrames for Iterative Workloads

    Learn how to cache DataFrames in Databricks to speed up iterative workloads, with hands-on steps and troubleshooting tips.

  8. 44
    Temporary vs Global Views

    Manage temporary views and global views in Databricks — learn the core concepts, step-by-step usage, hands-on exercise, troubleshooting, and what's next. Perfect for developers building skills incrementally.

  9. 45
    Use UDFs Sparingly

    Use UDFs and Pandas UDFs sparingly in Databricks. This lesson explains why native Spark functions perform better and guides you through hands-on steps to minimize UDF usage, plus troubleshooting and next steps.

    Databricks — step-by-step tutorials

    What you will find here

    This track walks through databricks in order — each lesson is server-rendered HTML you can read without JavaScript. Follow the outline, then practice in the browser IDE when a lesson links to runnable code.

    Tutorials vs quizzes and code samples

    Tutorials teach in sequence. For quick checks use quizzes. For copy-paste snippets see code samples. For deeper reading browse articles.