How to Implement Keyset Pagination in Python (Seek Method)
Implement keyset (seek) pagination in Python with a mock paginator that efficiently fetches pages based on the last row rather than OFFSET.
Python code
55 linesfrom dataclasses import dataclass
from typing import List, Optional
@dataclass
class Row:
id: int
name: str
def __lt__(self, other: "Row") -> bool:
return (self.id, self.name) < (other.id, other.name)
class MockKeysetPaginator:
"""Pagination using keyset (seek) method instead of OFFSET."""
def __init__(self, rows: List[Row], page_size: int = 3):
self.rows = sorted(rows)
self.page_size = page_size
def fetch_page(self, after: Optional[Row] = None) -> List[Row]:
"""Return page of rows that come after the given key."""
if after is None:
start_index = 0
else:
start_index = next(
(i for i, row in enumerate(self.rows) if row > after),
len(self.rows),
)
end_index = start_index + self.page_size
return self.rows[start_index:end_index]
if __name__ == "__main__":
data = [
Row(3, "alice"),
Row(1, "bob"),
Row(2, "carol"),
Row(5, "dave"),
Row(4, "eve"),
Row(6, "frank"),
Row(7, "grace"),
]
paginator = MockKeysetPaginator(data, page_size=3)
cursor = None
total_pages = 0
while True:
page = paginator.fetch_page(cursor)
if not page:
break
total_pages += 1
print(f"Page {total_pages}: {[row.name for row in page]}")
cursor = page[-1]
print(f"Total pages fetched: {total_pages}")
Output
Page 1: ['alice', 'bob', 'carol']
Page 2: ['dave', 'eve', 'frank']
Page 3: ['grace']
Total pages fetched: 3
How it works
Keyset pagination (seek method) uses the last fetched row as a cursor to find the next page, avoiding the performance penalty of OFFSET which re-scans skipped rows. The Row class defines a comparison operator (__lt__) so rows can be sorted by a composite key (id, name). fetch_page uses a generator expression to find the first index where the row exceeds the cursor, then slices the next page_size rows. This approach is O(n) per fetch but avoids the expensive database-side OFFSET scans in real production setups. The mock paginator demonstrates the core logic that can be translated into SQL queries with WHERE (id, name) > (?, ?) ORDER BY ... LIMIT ?.
Common mistakes
- Using `after` row only for comparison without tuple-based comparison, leading to incorrect ordering
- Forgetting to sort rows before paginating, causing unpredictable page content
- Not handling the case where the cursor equals the last row, returning an empty page
- Using `>=` instead of `>` in comparison, causing duplicate rows between pages
Variations
- Use SQL query with `WHERE (id, name) > %s ORDER BY id, name LIMIT %s` for database pagination
- Implement keyset pagination using an integer offset with a `LIMIT/OFFSET` for small datasets
Real-world use cases
- Fetching paginated search results from an analytics dashboard where users page through thousands of records.
- Querying a time-series database to load sensor readings in chronological order across API endpoints.
- Building a social media feed endpoint that streams posts for infinite scrolling without skipping items.
Sponsored
More from Database scaling & optimization
- Approximate Count with HyperLogLog in Python medium
- B-Tree Insert and In-Order Traversal in Python hard
- Broadcast a Small Reference Table in Python easy
- Build a Full Text Search Index in Python medium
- Build a Partial Index Mock in Python for Database Filtering easy
- Composite index leftmost prefix in Python medium
Keep learning
Related tutorials and quizzes for this topic.