Database indexing and query timing optimization in Python

Create SQLite indexes and time query performance to measure speedup for large table lookups in Python.

Easy Python 3.9+ Aug 9, 2026 Database scaling & optimization 14 views 0 copies

Python code

46 lines
Python 3.9+
import sqlite3
import time


def time_query(db_path, query, params=()):
    conn = sqlite3.connect(db_path)
    conn.execute("PRAGMA journal_mode = WAL")
    start = time.perf_counter()
    result = conn.execute(query, params).fetchall()
    elapsed = time.perf_counter() - start
    conn.close()
    return result, elapsed


def add_index(db_path, table, column):
    conn = sqlite3.connect(db_path)
    index_name = f"idx_{table}_{column}"
    conn.execute(f"CREATE INDEX IF NOT EXISTS {index_name} ON {table}({column})")
    conn.commit()
    conn.close()
    return index_name


if __name__ == "__main__":
    db = "scaling_optimization.db"
    conn = sqlite3.connect(db)
    conn.execute("DROP TABLE IF EXISTS users")
    conn.execute("CREATE TABLE users (id INTEGER, email TEXT, age INTEGER)")
    conn.executemany(
        "INSERT INTO users VALUES (?, ?, ?)",
        [(i, f"user{i}@example.com", 18 + (i % 50)) for i in range(10_000)],
    )
    conn.commit()
    conn.close()

    slow_query = "SELECT * FROM users WHERE email = ?"
    slow_result, slow_time = time_query(db, slow_query, ("user5000@example.com",))

    index_name = add_index(db, "users", "email")

    fast_result, fast_time = time_query(db, slow_query, ("user5000@example.com",))

    print(f"Slow query: {slow_time:.6f}s -> {slow_result}")
    print(f"Index created: {index_name}")
    print(f"Fast query: {fast_time:.6f}s -> {fast_result}")
    print(f"Speedup: {slow_time / fast_time:.2f}x")

Output

stdout
Slow query: 0.002319s -> [(5000, 'user5000@example.com', 18)]
Index created: idx_users_email
Fast query: 0.000142s -> [(5000, 'user5000@example.com', 18)]
Speedup: 16.33x

How it works

The time_query function sets WAL mode for better concurrency and measures query duration with time.perf_counter, which is precise on all platforms. The add_index helper creates a covering index using CREATE INDEX IF NOT EXISTS, avoiding errors on repeated runs. With 10,000 rows where the email is unique, the index turns a full table scan into a B-tree lookup, drastically reducing time. The script demonstrates that indexing a frequently queried column yields significant performance improvements, ideal for beginner-facing database optimization lessons.

Common mistakes

  • Forgetting `conn.commit()` after DDL statements like CREATE INDEX
  • Not closing connections, which can cause lock contention
  • Assuming indexes always help — they add write overhead

Variations

  1. Use `EXPLAIN QUERY PLAN` to verify the database uses the index
  2. Wrap timing logic in a context manager for cleaner code
  3. Apply composite indexes for multi-column WHERE clauses

Real-world use cases

  • Benchmarking query performance after adding indexes in application databases like PostgreSQL.
  • Evaluating whether a new index improves search speed in a large user table for an e-commerce platform.
  • Teaching QA or junior developers how to measure and document database query improvements.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Database scaling & optimization

Related tutorials and quizzes for this topic.