Using a Python Generator Instead of a List to Save Memory

Compare a list approach with a generator to stream values lazily, avoiding memory-heavy storage of large sequences.

Easy Python 3.9+ Aug 9, 2026 Concurrency & performance 12 views 0 copies

generator lazy-evaluation memory fibonacci streaming

Python code

27 lines

Python 3.9+

def fibonacci_generator(limit):
    a, b = 0, 1
    count = 0
    while count < limit:
        yield a
        a, b = b, a + b
        count += 1


def sum_first_n(generator, n):
    total = 0
    for i, value in enumerate(generator):
        if i >= n:
            break
        total += value
    return total


if __name__ == "__main__":
    # Using a generator avoids storing the entire sequence in memory
    fib_gen = fibonacci_generator(10)
    print("First 10 Fibonacci numbers:", list(fib_gen))
    
    # Create a new generator since the previous one is exhausted
    fib_gen_2 = fibonacci_generator(1000000)
    result = sum_first_n(fib_gen_2, 10)
    print("Sum of first 10:", result)

Output

stdout

First 10 Fibonacci numbers: [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
Sum of first 10: 88

How it works

The fibonacci_generator uses yield to produce values one at a time without storing the whole sequence. When you call list() on the generator, only then are the first 10 values materialized into memory. The second generator streams a million numbers without ever holding more than the current pair of values, thanks to lazy evaluation. The sum_first_n function consumes the stream on demand and stops early, proving you can process huge sequences with constant memory.

Common mistakes

Calling `list()` on a large generator defeats the memory-saving purpose
Reusing an exhausted generator without creating a new one — a generator can only be iterated once
Forgetting that `yield` is what makes a function a generator, not returning a value

Variations

Use `itertools.islice(generator, n)` to grab the first n elements in one line
Replace the manual `sum_first_n` with `sum(itertools.islice(gen, n))` for a compact version

Real-world use cases

Streaming sequential rows from a massive CSV database export without loading the file into RAM.
Rewriting a paginated API client to yield pages one at a time so each HTTP response is processed instantly.
Feeding a model training loop incremental batches from a never-ending sensor feed, holding only a window of data.

Using a Python Generator Instead of a List to Save Memory

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Concurrency & performance

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Concurrency & performance

Keep learning

Tutorials

Quizzes