How to Lazily Transform Items in Python with a Generator
Map a transform function over an iterable lazily with a generator so items are processed on demand, not up front.
Python code
24 linesdef lazy_map(items, transform):
for item in items:
yield transform(item)
def double(x):
return x * 2
def upper(s):
return s.upper()
if __name__ == "__main__":
numbers = [1, 2, 3, 4, 5]
doubled = lazy_map(numbers, double)
print("Doubled numbers:", end=" ")
for value in doubled:
print(value, end=" ")
print()
words = ["apple", "banana", "cherry"]
uppercased = lazy_map(words, upper)
print("Uppercased words:", end=" ")
for word in uppercased:
print(word, end=" ")
print()
Output
Doubled numbers: 2 4 6 8 10
Uppercased words: APPLE BANANA CHERRY
How it works
The lazy_map function is a generator because it contains a yield statement. Each call to next() on the generator runs the loop body until the next yield, computes transform(item), and pauses. This means transform is applied one item at a time only as you iterate, so memory usage stays flat even for huge inputs. The function works with any iterable and any callable, keeping the mapping logic reusable. This mirrors the behavior of Python's built-in map, which also produces items lazily.
Common mistakes
- Forgetting that a function with `yield` returns a generator object, not a list, so you can't index into it
- Calling `list()` on the generator too early and losing the memory benefit
- Assuming the transformation runs when `lazy_map` is called — it only runs during iteration
Variations
- Use the built-in `map(transform, items)` instead of a custom generator
- Write a generator expression: `(transform(item) for item in items)`
Real-world use cases
- Processing rows from a large CSV file one at a time to avoid loading the whole file into memory at once.
- Chaining lazy transforms on infinite or streaming data like a live sensor feed without exhausting RAM.
- Applying an expensive transformation only to items actually accessed, e.g., in a paginated API response handler.
Sponsored
More from Comprehensions & generators
- Batch Rows in Chunks with a Generator in Python easy
- Build a Generator Pipeline in Python: Filter Then Map medium
- Build a lazy generator to read file lines in Python easy
- Chunk an Iterable into Batches with a Generator in Python easy
- Convert Data in Python with Comprehensions and Generators easy
- Count Data in Python with Comprehensions and Generators easy
Keep learning
Related tutorials and quizzes for this topic.