Create Data Helper Functions in Python for Beginners
Build reusable Python helper functions to load, filter, sort, summarize, and save JSON data — a beginner-friendly starting point for small data pipelines.
Python code
63 linesimport json
from pathlib import Path
from typing import Any, Dict, List
def load_json_file(filepath: str) -> Dict[str, Any]:
"""Load JSON data from a file."""
with Path(filepath).open("r", encoding="utf-8") as file:
return json.load(file)
def filter_by_key(
data: List[Dict[str, Any]], key: str, value: Any
) -> List[Dict[str, Any]]:
"""Return records where data[key] equals value."""
return [record for record in data if record.get(key) == value]
def sort_by_key(
data: List[Dict[str, Any]], key: str, reverse: bool = False
) -> List[Dict[str, Any]]:
"""Sort records by a given key."""
return sorted(data, key=lambda record: record.get(key, 0), reverse=reverse)
def summary_stats(data: List[Dict[str, Any]], key: str) -> Dict[str, float]:
"""Return min, max, and mean of numeric values under key."""
values = [record[key] for record in data if isinstance(record.get(key), (int, float))]
if not values:
return {"min": None, "max": None, "mean": None}
return {
"min": min(values),
"max": max(values),
"mean": sum(values) / len(values),
}
def save_json_file(filepath: str, data: Any) -> None:
"""Save data to a JSON file."""
with Path(filepath).open("w", encoding="utf-8") as file:
json.dump(data, file, indent=2)
if __name__ == "__main__":
sample_data = [
{"name": "Alice", "score": 85},
{"name": "Bob", "score": 92},
{"name": "Charlie", "score": 78},
{"name": "Diana", "score": 92},
]
high_scorers = filter_by_key(sample_data, "score", 92)
sorted_by_score = sort_by_key(sample_data, "score", reverse=True)
stats = summary_stats(sample_data, "score")
print("High scorers (score=92):")
print(json.dumps(high_scorers, indent=2))
print("\nSorted by score descending:")
print(json.dumps(sorted_by_score, indent=2))
print("\nSummary statistics for score:")
print(json.dumps(stats, indent=2))
Output
High scorers (score=92):
[
{
"name": "Bob",
"score": 92
},
{
"name": "Diana",
"score": 92
}
]
Sorted by score descending:
[
{
"name": "Bob",
"score": 92
},
{
"name": "Diana",
"score": 92
},
{
"name": "Alice",
"score": 85
},
{
"name": "Charlie",
"score": 78
}
]
Summary statistics for score:
{
"min": 78,
"max": 92,
"mean": 86.75
}
How it works
The load_json_file function uses pathlib.Path to open and parse a JSON file into a Python dictionary or list, using json from the standard library. filter_by_key uses a list comprehension to keep only records where the given key matches the value, with .get() to safely handle missing keys. sort_by_key uses the sorted built-in with a lambda that extracts the key value; note records without the key default to 0 for sorting. summary_stats filters values to only numeric types (int or float) before computing min, max, and mean, and returns None placeholders if no numeric values exist. Finally, save_json_file writes data back to disk with pretty-printing via indent=2.
Common mistakes
- Forgetting that `.get()` returns `None` when a key is missing, which can cause `TypeError` in sorting or stats.
- Assuming all values under a key are numeric — the stats function skips non-numbers but others might not.
- Mixing up `json.load` (file) with `json.loads` (string).
- Not using `encoding='utf-8'` when opening files, which can lead to encoding errors on some platforms.
Variations
- Use `with open(filepath, 'r') as f: data = json.load(f)` instead of `pathlib.Path`.
- For larger datasets, use `pandas.read_json()` for more powerful filtering and aggregation.
Real-world use cases
- Process JSON API responses and filter records for downstream analytics in an ETL job.
- Load configuration files and compute summary stats for monitoring dashboard metrics.
- Clean and prepare exported JSON datasets by sorting and filtering before loading into a database.
Sponsored
More from Data pipelines & processing
- Add a UUID Surrogate Key to Each Row in a CSV with Python easy
- Attach Source File Metadata to Records in Python easy
- Build a Python Utility That Detects Duplicate Records Across Multiple Excel Sheets medium
- Check Null Rate Threshold in PySpark DataFrame medium
- Count Records Processed per Category in Python easy
- Deduplicate events by ID within a window in Python medium
Keep learning
Related tutorials and quizzes for this topic.