Temporary vs Global Views

Picture this: you've just spent twenty minutes crafting a perfect Spark DataFrame transformation in your Databricks notebook. You run the cell, it works beautifully, and then you try to reference that DataFrame from another notebook only to hit a NameError: name 'df' is not defined. Or worse, you register a temporary view, run a SQL query against it, and then watch it silently vanish when your job retries a failed step. These are the classic frustrations that plague every Databricks developer who hasn't yet mastered managing temporary views and global views. This lesson is your escape hatch — by the end, you'll know exactly how to create, scope, and drop these views so your data flows predictably across notebooks and jobs.

The problem this lesson solves

When you work in Databricks, you're not just writing code — you're defining a session of work. A Spark session has a lifecycle, and everything you create inside that session is ephemeral by default. Temporary views are tied to the SparkSession and disappear the moment that session ends. Global views, on the other hand, exist across sessions within the same cluster but still vanish when the cluster stops.

If you don't understand this scoping, you'll hit walls like:

• A notebook runs fine in development but fails in a production job because the view wasn't created in the same session.
• You try to access a view from a different notebook and get a Table or view not found error.
• You overwrite a view accidentally, silently breaking downstream queries that depended on the old definition.

These aren't edge cases — they're daily realities for teams sharing notebooks and orchestrating multi-step pipelines. Understanding how to manage temporary views and global views is the difference between brittle code and a robust data workflow.

Core concept / mental model

Think of a Spark session as a private workspace. Inside that workspace, you have two kinds of scratchpads:

• Temporary view (session-scoped): A private sticky note. Only you can see it, and it's torn down when you leave the room (the session ends).
• Global view (cluster-scoped): A whiteboard in the shared conference room. Anyone in the building (any session on the same cluster) can see it, but it's erased when the building closes (the cluster terminates).

Both types persist metadata only in memory — not in the metastore. That's a critical distinction from permanent tables, which are stored in Delta Lake and survive cluster restarts.

Here's the official naming convention you'll need to remember:

In Spark SQL, CREATE TEMP VIEW creates a session-scoped view, while CREATE GLOBAL TEMP VIEW creates a cluster-scoped view. Global views live in the global_temp database. When using the DataFrame API, .createOrReplaceTempView() is your friend for temporary views, and .createOrReplaceGlobalTempView() for global views.

Here's a quick mental mapping:

How it works step by step

Step 1: Create a temporary view

When you call df.createOrReplaceTempView("my_view"), Spark registers a plan for that DataFrame so you can query it using SQL in the same notebook and same SparkSession. If you're using SQL directly, you'd write:

CREATE OR REPLACE TEMP VIEW my_view AS
SELECT * FROM raw_data

The TEMP VIEW automatically inherits the current session's catalog, so you can reference the view by its unqualified name.

Step 2: Create a global temp view

For global views, the syntax shifts slightly because they live in a special database:

CREATE OR REPLACE GLOBAL TEMP VIEW my_global_view AS
SELECT * FROM raw_data

Then, when you query it from another session, you must fully qualify it:

SELECT * FROM global_temp.my_global_view

The same logic applies to the DataFrame API.

Step 3: Drop or replace views

Replacing is typically safer than dropping and recreating because it's atomic — the old definition is swapped out for the new one without a window where the view doesn't exist. If you must drop, use:

DROP VIEW IF EXISTS my_view;
DROP VIEW IF EXISTS global_temp.my_global_view;

Step 4: Understand the session lifecycle

Every Databricks notebook cell runs within a single SparkSession when using the default %sql or %python commands with the same cluster. However, if you detach and reattach a notebook, or if Databricks restarts the cluster, all temporary views are lost. Global views survive cluster restarts as long as the cluster itself doesn't stop, but they also rely on the same session if they're used within a single notebook.

Pro tip: Use spark.catalog.listTables() in Python to see the views and tables available in the current catalog and session.

Hands-on walkthrough

Let's bring this to life with a complete example. We'll create a sample DataFrame, register it as both a temporary and a global view, and then query them.

Python notebook example

from pyspark.sql import SparkSession

# SparkSession is already available in a Databricks notebook as 'spark'
# Create a simple DataFrame
sales_data = [
    ("2024-01-01", "North", 100),
    ("2024-01-01", "South", 150),
]
columns = ["date", "region", "revenue"]
df = spark.createDataFrame(sales_data, columns)

# Register as a temporary view (session-scoped)
df.createOrReplaceTempView("sales_temp")

# Register as a global temp view (cluster-scoped)
df.createOrReplaceGlobalTempView("sales_global")

# Query the temporary view using SQL
spark.sql("SELECT region, SUM(revenue) FROM sales_temp GROUP BY region").show()

Expected output:

+------+-------------+
|region|sum(revenue)|
+------+-------------+
| South|          150|
| North|          100|
+------+-------------+

Now, from a different notebook on the same cluster, you can access only the global view:

# In a separate notebook (same cluster, different session)
spark.sql("SELECT * FROM global_temp.sales_global").show()

Expected output:

+----------+------+-------+
|      date|region|revenue|
+----------+------+-------+
|2024-01-01| North|    100|
|2024-01-01| South|    150|
+----------+------+-------+

If you tried to query sales_temp from that second notebook, you'd get a Table or view not found error — exactly the pain we set out to solve.

SQL-only example

You can also do everything in SQL cells:

-- Create a global temp view
CREATE OR REPLACE GLOBAL TEMP VIEW sales_by_region AS
SELECT region, SUM(revenue) AS total_revenue
FROM sales
GROUP BY region;

-- Query it (must include global_temp prefix)
SELECT * FROM global_temp.sales_by_region ORDER BY total_revenue DESC;

Listing and dropping views

# List all views and tables in the current catalog
print("Current catalog:", spark.catalog.currentCatalog())
print("Tables:", spark.catalog.listTables())

# Drop a temporary view
spark.catalog.dropTempView("sales_temp")
print("After drop:", [t.name for t in spark.catalog.listTables() if t.name == "sales_temp"])

Expected output:

Current catalog: spark_catalog
Tables: [Table(name='sales_global', ...), Table(name='sales_temp', ...)]
After drop: []

Compare options / when to choose what

Now that you've seen both in action, here's the decision framework:

When to use what:

• Use a temporary view when the data is only needed within the current notebook and session. It's the lightest option and keeps your namespace clean.
• Use a global view when you need to share a DataFrame across notebooks that run on the same cluster but in different sessions. Common in multi-notebook pipelines where a shared preparation step feeds multiple consumers.
• Use a permanent table when the data must survive cluster shutdowns or be accessed by multiple clusters — then you're moving into Delta Lake territory.

Troubleshooting & edge cases

Error: Table or view not found when referencing a temporary view from another notebook

This is the most common pitfall. Remember: temporary views are session-scoped. If you see this, switch to a global view or persist the data.

Error: Cannot find table 'global_temp.my_view' when using the DataFrame API

You must use the fully qualified name global_temp.my_view in SQL. The same applies when you access it via the catalog. If you used CREATE GLOBAL TEMP VIEW, the view is physically placed in the global_temp database.

Overwriting a view accidentally

If two notebooks on the same cluster both create a global view with the same name, the last writer wins. This can silently break downstream processing. Use CREATE OR REPLACE consciously, and consider naming conventions like project_step_view.

Edge case: DEFAULT database changes

If you run USE my_catalog.my_database; in a notebook, temporary views are still session-scoped, but global views remain in global_temp regardless of the current database, so always reference them with the prefix.

Spark UI vs. catalog

Remember that views are logical, not physical. You won't see them in the Databricks Explorer under the default database unless you list tables within a session. To see them, use spark.catalog.listTables() or the UI's schema tab.

What you learned & what's next

You now have a solid grasp of how to manage temporary views and global views in Databricks. Let's recap the essentials:

• Temporary views are session-scoped and ideal for single-notebook use.
• Global views are cluster-scoped and accessible from any notebook on the same cluster via the global_temp schema.
• Both are ephemeral — they die with the cluster.
• Use CREATE OR REPLACE to swap definitions safely.
• List and drop views easily with catalog APIs.

You've seen how this fits into a larger data pipeline: temporary views help you iterate quickly, global views let you share prepared data across notebooks, and permanent Delta tables give you durability. Your next step in the Databricks track is to explore managing permanent tables and Delta Lake — where you’ll learn to make your data persist beyond a single cluster and unlock transactions, time travel, and schema enforcement.