PyMongo Cursor Methods

Learn to use PyMongo cursor methods for advanced querying in MongoDB—sorting, limiting, skipping, and projecting data efficiently. Includes hands-on steps and troubleshooting.

Focus: use pymongo cursor methods for advanced querying

Sponsored

You've got your PyMongo queries returning documents, but your results are a messy wall of data — unsorted, unlimited, and pulling every field you don't need. You need to sort, paginate, and project with precision, and you're about to discover that find() alone isn't the whole story. The real power lies in the cursor object it returns and the chainable methods that let you shape your query results exactly how you want them.

The problem this lesson solves

When you call collection.find(), PyMongo doesn't return a list of documents. It returns a cursor — a lazy, iterable object that fetches documents from MongoDB in batches. Beginners often treat the cursor as a simple list, but that's where the trouble begins. Without using cursor methods, you're stuck with default behavior: documents in natural order, all fields included, and no way to paginate or limit results beyond what the server decides.

This becomes a real pain in production. Your analytics dashboard loads thousands of documents when users only need the top 10. Your API returns entire user profiles including password hashes and internal notes when clients only need names and emails. And your pagination breaks because you're relying on skip() without a deterministic sort order. The problem isn't that MongoDB can't handle your queries — it's that you're not using the cursor methods that make advanced querying possible.

Core concept / mental model

Think of a cursor as a pointer into the result set of your query. When you run find(), MongoDB identifies matching documents, but the cursor lets you navigate through them with methods that transform or restrict that set before you iterate.

The cursor as a pipeline

Imagine a factory conveyor belt carrying documents. The cursor is your control panel. You can:

  • Sort the belt by a field (like price or date) using .sort()
  • Skip the first few items with .skip()
  • Limit how many items you inspect with .limit()
  • Project which fields to keep using the projection parameter in find()

You chain these methods together — like a pipeline — to get exactly the documents you need. The beauty is that these operations are executed server-side, so you're not downloading everything to your Python process just to filter locally.

Definitions

  • Cursor: A lazy iterator over query results. It doesn't fetch all documents at once.
  • Sort: Orders documents by one or more fields, ascending (1) or descending (-1).
  • Skip: Discards the first N documents from the result set.
  • Limit: Caps the total number of documents returned.
  • Projection: A document specifying which fields to include (1) or exclude (0).

How it works step by step

Step 1: Get a cursor with find()

Every find() call returns a cursor. Even without any filter, you're working with a cursor object. Important: The cursor is lazy — it doesn't hit the server until you iterate over it (e.g., with a for loop or list()).

Step 2: Chain methods on the cursor

sort(), skip(), and limit() are all methods of the cursor object. They return the cursor itself (or a new cursor in some cases), so you can chain them. The order of chaining matters for readability but not for correctness — PyMongo applies them in a consistent server-side fashion.

Step 3: Use projections to control fields

Projection is passed as a second argument to find(), not as a cursor method. It's part of the query specification. You include fields with 1 (true) or exclude with 0 (false). You can't mix include and exclude — except for _id, which you can exclude even when including others.

Step 4: Iterate the cursor

Once you've applied your methods, you iterate. The cursor fetches documents in batches (e.g., 101 docs at a time). When you exhaust a batch, it fetches the next. If you call list(cursor), you pull everything into memory — use with caution.

Hands-on walkthrough

Let's build a sample collection and exercise the cursor methods. We'll simulate a product inventory.

Setup

First, create a test database and insert some documents.

from pymongo import MongoClient
from pymongo import DESCENDING, ASCENDING

client = MongoClient("mongodb://localhost:27017")
db = client["shop"]
products = db["products"]

# Clear existing data for a clean run
products.delete_many({})

# Insert sample products
products.insert_many([
    {"name": "Laptop", "price": 1200, "stock": 15, "category": "electronics"},
    {"name": "Mouse", "price": 25, "stock": 200, "category": "accessories"},
    {"name": "Keyboard", "price": 75, "stock": 80, "category": "accessories"},
    {"name": "Monitor", "price": 300, "stock": 12, "category": "electronics"},
    {"name": "Desk", "price": 450, "stock": 5, "category": "furniture"},
    {"name": "Chair", "price": 150, "stock": 30, "category": "furniture"},
])
print("Inserted 6 products")

Sort, limit, and project

Now let's retrieve the top 3 cheapest products, showing only name and price.

cursor = products.find(  # filter
    {},
    {"_id": 0, "name": 1, "price": 1}  # projection: exclude _id, include name and price
).sort("price", ASCENDING).limit(3)

for doc in cursor:
    print(doc)

Expected output:

{'name': 'Mouse', 'price': 25}
{'name': 'Keyboard', 'price': 75}
{'name': 'Chair', 'price': 150}

Paginate with skip and limit

For a web API, you'd implement pagination. Here's page 2 (items 4–6) sorted by price:

page = 2
page_size = 3

cursor = (products.find({})
          .sort("price", ASCENDING)
          .skip((page - 1) * page_size)
          .limit(page_size))

for doc in cursor:
    print(doc["name"], doc["price"])

Expected output:

Monitor 300
Desk 450
Laptop 1200

Pro tip: For large collections, skip() can be slow because MongoDB must scan and discard documents. For big datasets, prefer range-based pagination using a unique field like _id or a timestamp.

Sort by multiple fields

Often you need to sort by several criteria. Let's sort by category (alphabetical) and then price (descending) within each category.

cursor = (products.find({})
          .sort([("category", ASCENDING), ("price", DESCENDING)])
          .limit(5))

for doc in cursor:
    print(doc["category"], doc["name"], doc["price"])

Expected output:

accessories Keyboard 75
accessories Mouse 25
electronics Laptop 1200
electronics Monitor 300
furniture Desk 450

Notice the sort takes a list of (field, direction) tuples.

Count without loading documents

If you just need the number of matching documents, use count_documents() on the collection, not on the cursor.

count = products.count_documents({"category": "electronics"})
print(f"Electronics count: {count}")

Expected output:

Electronics count: 2

Compare options / when to choose what

Let's compare common cursor methods and their use cases.

Method Purpose When to use
.sort() Order documents Any time order matters (e.g., newest first, cheapest first)
.limit() Cap result size Pagination, top-N widgets, limiting bandwidth
.skip() Offset the start Simple pagination (but see caveats)
projection (in find()) Select fields When you need only a subset of fields, or to hide sensitive data
.count_documents() Count matches When you need a count without fetching documents
.distinct() Unique values Getting distinct values for a field (e.g., categories)

When to not use cursor methods

  • Avoid .skip() for deep pagination on huge collections — use _id ranges.
  • Avoid .sort() on unindexed fields for large collections — create indexes.
  • Avoid pulling all documents into memory with list(cursor) if you can iterate.

Troubleshooting & edge cases

Issue: TypeError: cannot mix inclusion and exclusion in projection

You tried to include some fields and exclude others (except _id). MongoDB doesn't allow mixing 1 and 0 in a projection. Choose one: either list includes or list excludes.

Fix: Use all 1s (include) or all 0s (exclude). If you exclude _id, you can still include others.

# Correct: include name and price, exclude _id
cursor = products.find({}, {"_id": 0, "name": 1, "price": 1})

# Incorrect: mixing include (name) and exclude (stock)
# cursor = products.find({}, {"name": 1, "stock": 0})

Issue: Cursor already started or exhausted

If you try to call .sort() after iterating over the cursor, you'll get a TypeError: cannot set options after executing query. The cursor can't be modified once it's been consumed.

Fix: Chain all methods before iterating. If you need a new query, create a new cursor.

Issue: Slow pagination with skip()

Using skip() with large offsets (e.g., skip(10000)) is inefficient because MongoDB must scan and discard those documents.

Fix: Use range-based pagination: filter on _id greater than the last _id from the previous page and use limit().

# Get first page
last_id = None
page_size = 10

while True:
    query = {"_id": {"$gt": last_id}} if last_id else {}
    cursor = products.find(query).sort("_id", 1).limit(page_size)
    docs = list(cursor)
    if not docs:
        break
    # process docs...
    last_id = docs[-1]["_id"]

Issue: Sort on unindexed field

Sorting a large collection on a field without an index causes MongoDB to do an in-memory sort, which can hit the 32MB limit and throw an error.

Fix: Create an index on the sort field using products.create_index("price") or compound indexes for multiple fields.

What you learned & what's next

You've mastered the core cursor methods in PyMongo: .sort(), .skip(), .limit(), and projections in find(). You now understand that cursors are lazy pipelines that let you shape your query results efficiently. You also learned how to count documents without fetching them, and you're aware of common pitfalls like mixing projections and slow skips.

Next step: Now that you can control the shape and order of query results, the next lesson in this track will teach you how to aggregate and group data using MongoDB's aggregation pipeline. You'll combine these cursor techniques with powerful stage operations like $group and $match to build complex analytics queries. Get ready to transform raw data into meaningful insights!

Practice recap

Now practice with a real collection: insert 20 documents, then write a query that returns the 5 cheapest products sorted by price, projecting only name and price. Then implement a pagination loop using _id ranges instead of skip(). Test with a large dataset (e.g., 100k documents) to observe performance differences.

Common mistakes

  • Mixing inclusion and exclusion in a projection (except for _id) causes a TypeError. Use either all 1s or all 0s.
  • Calling .sort() or .limit() after you've already started iterating raises an error. Chain all methods before looping.
  • Heavy reliance on .skip() for deep pagination — it's slow and can be replaced with range-based pagination using a unique field like _id.
  • Forgetting that .count_documents() is not a cursor method; calling .count() on a cursor is deprecated and less efficient.
  • Pulling an entire result set into memory with list(cursor) when you only need to process one document at a time, potentially exhausting memory.

Variations

  1. Use $sort inside the aggregation pipeline when combining sorting with grouping or projections that can't be done with a simple find().
  2. For extremely large datasets, use range-based pagination (_id > last_id) instead of skip()/limit().
  3. Consider using MongoDB's native sort() with an index that matches your sort pattern to avoid in‑memory sorts.

Real-world use cases

  • Implement server-side pagination for an e-commerce search API, sorting by price and applying limit/skip for page navigation.
  • Return only public fields (name, price, description) for a product listing while hiding internal fields like cost or inventory metadata in an admin API.
  • Generate a 'Top 10 best-selling products' report by sorting a cursor on total sales in descending order and limiting to 10 results.

Key takeaways

  • A PyMongo cursor is a lazy pipeline for query results, not a simple list; always apply cursor methods before iterating.
  • Use .sort(), .skip(), and .limit() to control order, offset, and size of query results server-side.
  • Projection in find() lets you include or exclude fields, but never mix inclusion and exclusion (except for _id).
  • For counting, use count_documents() on the collection, not a deprecated .count() on the cursor.
  • Deep pagination should use range-based queries on an indexed unique field instead of .skip() for performance.
  • Index your sort fields to avoid slow in-memory sorts that can exceed MongoDB's 32MB limit.

Sponsored

Sponsored

Discussion

Questions, corrections, and tips help everyone reading this page.

0 comments

Add a comment

Shown publicly with your comment.

Be constructive · max 4,000 characters

No comments yet — start the thread.

Related tutorials, quizzes, and articles for this topic.