Build a lazy generator to read file lines in Python

Create a generator function that yields file lines one at a time, avoiding loading the entire file into memory, and demonstrate its lazy processing.

Easy Python 3.9+ Aug 9, 2026 Comprehensions & generators 14 views 0 copies

generator file-io lazy

Python code

22 lines

Python 3.9+

def lazy_lines(filepath):
    """Yield lines from a file one at a time without loading the whole file into memory."""
    with open(filepath, 'r', encoding='utf-8') as file:
        for line in file:
            yield line.rstrip('\n')


if __name__ == "__main__":
    # Create a sample file to demonstrate
    sample_content = "first line\nsecond line\nthird line\nfourth line"
    with open("sample.txt", "w") as f:
        f.write(sample_content)

    # Use the generator lazily
    line_generator = lazy_lines("sample.txt")
    
    # Process lines one by one
    for idx, line in enumerate(line_generator, start=1):
        print(f"Line {idx}: {line}")
    
    # Show it's a generator object
    print(f"\nType: {type(line_generator)}")

Output

stdout

Line 1: first line
Line 2: second line
Line 3: third line
Line 4: fourth line

Type: <class 'generator'>

How it works

The lazy_lines function is a generator because it uses the yield keyword. It opens the file and iterates over each line, stripping the trailing newline before yielding it. Because it's a generator, it reads one line at a time from the file, so the whole file is never loaded into memory. This is efficient for large files. The with statement ensures the file is closed properly after iteration completes.

Common mistakes

Using `readlines()` instead of iterating directly, which loads the entire file into memory
Forgetting to strip newline characters, leaving `\n` in the yielded lines
Not closing the file manually when not using a context manager

Variations

Use `yield from file` to yield lines without stripping newlines
Use a list comprehension over the file object to create a list, but this is not lazy

Real-world use cases

Processing huge log files line-by-line in a memory-constrained environment.
Streaming data from a CSV file into a database without loading the entire dataset.
Implementing a custom file reader in a pipeline that processes records incrementally.

Build a lazy generator to read file lines in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Comprehensions & generators

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Comprehensions & generators

Keep learning

Tutorials

Quizzes