How to Filter Data in Python
Filter a list of dictionaries by exact key-value matches or numerical ranges using concise list comprehensions.
Python code
30 linesfrom typing import List, Dict, Any
def filter_data(
data: List[Dict[str, Any]], key: str, value: Any
) -> List[Dict[str, Any]]:
"""Return records where data[key] equals value."""
return [record for record in data if record.get(key) == value]
def filter_by_range(
data: List[Dict[str, Any]], key: str, low: float, high: float
) -> List[Dict[str, Any]]:
"""Return records where low <= data[key] <= high."""
return [record for record in data if low <= record.get(key) <= high]
if __name__ == "__main__":
dataset = [
{"name": "Alice", "age": 30, "city": "NYC"},
{"name": "Bob", "age": 25, "city": "LA"},
{"name": "Carol", "age": 35, "city": "NYC"},
{"name": "Dave", "age": 40, "city": "Chicago"},
]
print("Filter by city == 'NYC':")
print(filter_data(dataset, "city", "NYC"))
print("\nFilter age between 25 and 35:")
print(filter_by_range(dataset, "age", 25, 35))
Output
Filter by city == 'NYC':
[{'name': 'Alice', 'age': 30, 'city': 'NYC'}, {'name': 'Carol', 'age': 35, 'city': 'NYC'}]
Filter age between 25 and 35:
[{'name': 'Alice', 'age': 30, 'city': 'NYC'}, {'name': 'Bob', 'age': 25, 'city': 'LA'}, {'name': 'Carol', 'age': 35, 'city': 'NYC'}]
How it works
The filter_data function uses a list comprehension with record.get(key) == value to match exact field values. The record.get() method safely returns None when the key is missing, avoiding a KeyError. filter_by_range chains two comparisons — low <= value <= high — which Python evaluates as a single concise range check. Both functions return new lists, leaving the original dataset unchanged, which is ideal for immutable data pipelines. The __main__ guard lets the script run standalone while still being importable as a module.
Common mistakes
- Using `record[key]` instead of `.get()` and crashing on missing keys
- Forgetting that range filtering includes both endpoints (low and high)
- Mutating the original list instead of returning a filtered copy
Variations
- Use a generator expression and convert to a list for large datasets
- Parameterize the comparison operator (e.g., '=' vs '<') for flexible queries
Real-world use cases
- Filtering user records by a status field in an ETL job before loading into a warehouse.
- Selecting sensor readings within a healthy temperature range for anomaly detection.
- Filtering orders by location or price tier before sending to a recommendation engine.
Sponsored
More from Data pipelines & processing
- Add a UUID Surrogate Key to Each Row in a CSV with Python easy
- Attach Source File Metadata to Records in Python easy
- Build a Python Utility That Detects Duplicate Records Across Multiple Excel Sheets medium
- Check Null Rate Threshold in PySpark DataFrame medium
- Count Records Processed per Category in Python easy
- Create Data Helper Functions in Python for Beginners easy
Keep learning
Related tutorials and quizzes for this topic.