Build a Data Helper Class in Python for ML Pipelines
A beginner-friendly Python class that summarizes, filters, and exports ML dataset rows as JSON.
Python code
41 linesfrom typing import List, Dict, Any
import json
class DataHelper:
"""Beginner-friendly helpers for ML data pipelines."""
def __init__(self, data: List[Dict[str, Any]]):
self.data = data
self.keys = list(data[0].keys()) if data else []
def summary(self) -> Dict[str, Any]:
"""Return basic stats: row count, columns, missing values."""
missing = {k: sum(1 for row in self.data if not row.get(k)) for k in self.keys}
return {
"rows": len(self.data),
"columns": self.keys,
"missing_values": missing
}
def filter_by(self, key: str, value: Any) -> "DataHelper":
"""Return new DataHelper filtered by key-value match."""
filtered = [row for row in self.data if row.get(key) == value]
return DataHelper(filtered)
def to_json(self, filepath: str) -> None:
"""Save data to a JSON file."""
with open(filepath, "w") as f:
json.dump(self.data, f, indent=2)
if __name__ == "__main__":
sample = [
{"name": "alice", "age": 25, "score": 88},
{"name": "bob", "age": 30, "score": 0},
{"name": "carol", "age": 22, "score": 95}
]
helper = DataHelper(sample)
print(helper.summary())
adults = helper.filter_by("age", 30)
print(adults.data)
helper.to_json("output.json")
print("saved to output.json")
Output
{'rows': 3, 'columns': ['name', 'age', 'score'], 'missing_values': {'name': 0, 'age': 0, 'score': 1}}
[{'name': 'bob', 'age': 30, 'score': 0}]
saved to output.json
How it works
The DataHelper class wraps a list of dictionaries, which is the standard way Python represents tabular data rows. The summary method scans each column with row.get(k) to count falsy values like 0 or empty strings as missing, giving a quick data-quality check. filter_by applies a dictionary comprehension with a key-value match and returns a new DataHelper, keeping methods chainable and immutable. The to_json method persists the rows with indentation for readability. This pattern gives beginners a reusable, readable starting point for manual data inspection before heavier tools are introduced.
Common mistakes
- Assuming `row.get(k)` treats only None or NaN as missing, while 0, "", and False also count.
- Using `data[0]` without guarding against an empty dataset, raising IndexError.
- Forgetting to pass an `encoding` or closing the file explicitly when writing JSON on all platforms.
Variations
- Return plain dictionaries instead of a new DataHelper from `filter_by` for simpler consumers.
- Add pandas in `pip_requirements` and convert rows to a DataFrame for more advanced analysis.
Real-world use cases
- Quickly validating the shape and missing-value counts of a CSV before feeding it into a training job.
- Filtering rows by category or threshold in a preprocessing step of an ETL pipeline.
- Exporting a subset of labeled data to JSON for manual review or handoff to a labeling team.
Sponsored
More from ML engineering pipelines
- Bayesian Optimization in Python: A Simplified Mock Implementation medium
- Build a Mock Random Forest Classifier in Python easy
- Champion Challenger Deployment Mock in Python easy
- Compare Model A vs Model B Metrics in Python easy
- Create a Minimal Great Expectations Suite Mock in Python easy
- Detect Concept Drift in Python with a Simple Statistical Test medium
Keep learning
Related tutorials and quizzes for this topic.