How to Filter Data with Predicates in Python
This helper filters a list with a predicate using a list comprehension, plus a lazy generator version that yields matches one by one.
Python code
31 linesdef filter_data(data, predicate):
"""Return a list containing only items that pass the predicate."""
return [item for item in data if predicate(item)]
def filter_data_lazy(data, predicate):
"""Generator version: yields items that pass the predicate one by one."""
for item in data:
if predicate(item):
yield item
def is_even(number):
return number % 2 == 0
def is_long(word):
return len(word) > 4
if __name__ == "__main__":
numbers = [1, 2, 3, 4, 5, 6, 7, 8]
words = ["cat", "elephant", "dog", "tiger", "fox"]
even_numbers = filter_data(numbers, is_even)
long_words = filter_data(words, is_long)
lazy_even = list(filter_data_lazy(numbers, is_even))
print("Even numbers:", even_numbers)
print("Long words:", long_words)
print("Lazy evens:", lazy_even)Output
Even numbers: [2, 4, 6, 8]
Long words: ['elephant', 'tiger']
Lazy evens: [2, 4, 6, 8]How it works
The filter_data function uses a list comprehension to iterate over data and include only items where predicate(item) is truthy. The generator filter_data_lazy uses a for loop with yield to produce matches lazily, saving memory for large datasets. Both accept any callable predicate, making the helper reusable for any condition. The module-level is_even and is_long functions act as simple predicates, demonstrating how to pass functions as arguments. The if __name__ == "__main__" guard runs demo code only when executed directly, keeping the helper importable.
Common mistakes
- Passing a method call like `is_even()` instead of the function `is_even` as the predicate.
- Forgetting to convert a generator to a list when you need a concrete list, e.g., `list(filter_data_lazy(...))`.
- Using a mutable default argument for `data` or `predicate`, which can lead to unexpected shared state.
Variations
- Use the built-in `filter(predicate, data)` to get an iterator without a custom function.
- Write a lambda inline: `filter_data(numbers, lambda x: x % 2 == 0)` for simple one-off conditions.
Real-world use cases
- Filtering a list of user IDs to find active accounts before sending notifications.
- Extracting only valid log lines from a large log file in a streaming pipeline.
- Selecting relevant records from an API response where a status field meets a threshold.
Sponsored
More from Comprehensions & generators
- Batch Rows in Chunks with a Generator in Python easy
- Build a Generator Pipeline in Python: Filter Then Map medium
- Build a lazy generator to read file lines in Python easy
- Chunk an Iterable into Batches with a Generator in Python easy
- Convert Data in Python with Comprehensions and Generators easy
- Count Data in Python with Comprehensions and Generators easy
Keep learning
Related tutorials and quizzes for this topic.