How to Parse Data with Generators and Comprehensions in Python

This code demonstrates using a generator expression to filter active users and a dictionary comprehension to aggregate scores by name.

Easy Python 3.9+ Aug 9, 2026 Comprehensions & generators 15 views 0 copies

Python code

30 lines
Python 3.9+
def parse_data_helper(raw_records):
    """Extract active users' names and scores from raw records."""
    parsed = (
        (record["name"], record["score"])
        for record in raw_records
        if record["active"] and record["score"] >= 0
    )
    return list(parsed)


def aggregate_scores(parsed_data):
    """Sum scores grouped by name using a comprehension."""
    names = {name for name, _ in parsed_data}
    return {
        name: sum(score for n, score in parsed_data if n == name)
        for name in names
    }


if __name__ == "__main__":
    raw_data = [
        {"name": "Ana", "score": 10, "active": True},
        {"name": "Bob", "score": 5, "active": False},
        {"name": "Ana", "score": 7, "active": True},
        {"name": "Cal", "score": -3, "active": True},
    ]

    filtered = parse_data_helper(raw_data)
    print("Parsed:", filtered)
    print("Aggregated:", aggregate_scores(filtered))

Output

stdout
Parsed: [('Ana', 10), ('Ana', 7)]
Aggregated: {'Ana': 17}

How it works

The generator expression inside parse_data_helper lazily filters raw records by checking the active flag and non-negative score, yielding tuples of name and score. list(parsed) materializes the filtered results. aggregate_scores builds a set of unique names and then uses a dictionary comprehension to sum scores for each name, iterating over the list multiple times. This approach showcases readable, functional-style data transformation without explicit loops.

Common mistakes

  • Forgetting to wrap the generator expression in `list()` if you need to reuse it later.
  • Assuming the generator can be iterated multiple times — it's consumed after one pass.
  • Not handling missing dictionary keys, leading to `KeyError` if `'active'` or `'score'` is absent.

Variations

  1. Use a list comprehension instead of a generator if you don't need lazy evaluation.
  2. Use `itertools.groupby` on sorted data for more efficient aggregation.

Real-world use cases

  • Filtering and extracting user analytics from a JSON API response before storing in a database.
  • Aggregating sales totals by product category from a CSV import in an ETL pipeline.
  • Preprocessing feature matrices by selecting active records and summarizing per-group metrics.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Comprehensions & generators

Related tutorials and quizzes for this topic.