How to Parse CSV Rows as Generator Dicts in Python
Reads a CSV file and yields each row as a dictionary one at a time using a generator, so the file is processed lazily.
Python code
17 linesimport csv
from pathlib import Path
def csv_to_dicts(filepath):
with open(filepath, mode="r", newline="", encoding="utf-8") as file:
reader = csv.DictReader(file)
for row in reader:
yield row
if __name__ == "__main__":
sample_csv = Path("sample_data.csv")
sample_csv.write_text("name,age,city\nAlice,30,New York\nBob,25,London\nCarol,35,Paris\n", encoding="utf-8")
for person in csv_to_dicts(sample_csv):
print(person)
sample_csv.unlink()
Output
{'name': 'Alice', 'age': '30', 'city': 'New York'}
{'name': 'Bob', 'age': '25', 'city': 'London'}
{'name': 'Carol', 'age': '35', 'city': 'Paris'}
How it works
This function uses csv.DictReader to convert each row into a dictionary where the keys come from the header row. The generator yields each dictionary on demand, so the entire file is not loaded into memory at once — useful for large files. The newline="" argument prevents blank lines in the parsed output, and encoding="utf-8" ensures correct text decoding. Because the function is a generator, calling it returns a generator object that you can iterate over once. Using yield instead of building and returning a list keeps memory usage low even for millions of rows.
Common mistakes
- Forgetting to pass `newline=""` to `open` when reading CSV, which can cause blank lines in the output.
- Assuming the returned object is a list — it's a generator, so you must iterate it or call `list()` to get all rows.
- Hard-coding the header row instead of letting `csv.DictReader` derive it from the first line.
Variations
- Wrap the generator in `list()` if you need all rows at once: `rows = list(csv_to_dicts('file.csv'))`.
- Use `csv.reader` and manually zip with a header list if you need custom column mapping.
Real-world use cases
- Streaming large export files from a database into an ETL pipeline without exhausting memory.
- Reading customer records from a daily CSV dump and normalizing fields before inserting into a data warehouse.
- Processing CSV logs line-by-line to compute aggregates like average order value without loading the whole file.
Sponsored
More from Comprehensions & generators
- Batch Rows in Chunks with a Generator in Python easy
- Build a Generator Pipeline in Python: Filter Then Map medium
- Build a lazy generator to read file lines in Python easy
- Chunk an Iterable into Batches with a Generator in Python easy
- Convert Data in Python with Comprehensions and Generators easy
- Count Data in Python with Comprehensions and Generators easy
Keep learning
Related tutorials and quizzes for this topic.