Offset vs Keyset Pagination in Python

Demonstrate offset-based pagination and keyset (cursor) pagination with a simple in-memory dataset, showing how each returns pages of records.

Medium Python 3.9+ Aug 9, 2026 Database scaling & optimization 14 views 0 copies

Python code

36 lines
Python 3.9+
"""Demonstrate pagination using offset vs keyset (cursor) approach."""

ITEMS = [
    {"id": 1, "name": "Alice"},
    {"id": 2, "name": "Bob"},
    {"id": 3, "name": "Carol"},
    {"id": 4, "name": "David"},
    {"id": 5, "name": "Eve"},
]

def offset_paginate(items, page, page_size):
    """Return a page using offset-based pagination."""
    start = (page - 1) * page_size
    return items[start:start + page_size]

def keyset_paginate(items, last_id, page_size):
    """Return a page using keyset (cursor) pagination, assuming sorted by id."""
    if last_id is None:
        return items[:page_size]
    start = next(i for i, item in enumerate(items) if item["id"] == last_id) + 1
    return items[start:start + page_size]

if __name__ == "__main__":
    print("Offset pagination:")
    for page in range(1, 4):
        result = offset_paginate(ITEMS, page, page_size=2)
        print(f"  page {page}: {[item['name'] for item in result]}")

    print("Keyset pagination:")
    cursor = None
    for _ in range(3):
        result = keyset_paginate(ITEMS, cursor, page_size=2)
        if not result:
            break
        print(f"  after {cursor}: {[item['name'] for item in result]}")
        cursor = result[-1]["id"]

Output

stdout
Offset pagination:
  page 1: ['Alice', 'Bob']
  page 2: ['Carol', 'David']
  page 3: ['Eve']
Keyset pagination:
  after None: ['Alice', 'Bob']
  after 2: ['Carol', 'David']
  after 4: ['Eve']

How it works

Offset pagination calculates a slice by multiplying (page-1) * page_size to find the start index, then uses Python slicing to return that page. Keyset pagination tracks the last item's unique ID (cursor) and finds the next page by locating that ID and slicing from the following element. Offset is simple but can slow down on large tables because the database must skip many rows. Keyset is more efficient for large datasets because it uses indexed columns to jump directly to the next batch, avoiding costly OFFSET scans.

Common mistakes

  • Forgetting that offset pages can show duplicate or missing rows when new records are inserted between requests
  • Using keyset pagination on a column that is not unique or not sorted with a stable order
  • Assuming `items` is sorted by the cursor column; keyset requires a deterministic ordering to work correctly

Variations

  1. Use a database index (e.g., on `id`) and a WHERE clause like `WHERE id > last_id ORDER BY id LIMIT page_size` for keyset in SQL
  2. Return the next cursor in the API response so clients can pass it back for the next page request

Real-world use cases

  • Implementing a REST API endpoint that lists user records — offset is fine for small tables, keyset scales better for millions of rows.
  • Building an infinite scroll feed that needs stable ordering without page drift as new items are added.
  • Writing a nightly sync script that pages through a large database table without memory blow-ups on each request.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Database scaling & optimization

Related tutorials and quizzes for this topic.