How to Create a Simple Data Helper in Python for LLM Projects
Create a beginner-friendly Python class that stores, filters, and serializes data records for AI/LLM workflows.
Python code
42 linesimport json
from typing import Any, Dict, List, Optional
class DataHelper:
"""Simple helper for beginners to manage data in AI/LLM projects."""
def __init__(self, data: Optional[List[Dict[str, Any]]] = None) -> None:
self.data: List[Dict[str, Any]] = data or []
def add_item(self, item: Dict[str, Any]) -> None:
"""Add a single data item."""
self.data.append(item)
def to_json(self, indent: int = 2) -> str:
"""Serialize data to JSON string (common LLM input format)."""
return json.dumps(self.data, indent=indent)
def filter_by(self, key: str, value: Any) -> List[Dict[str, Any]]:
"""Filter records by a key-value pair."""
return [item for item in self.data if item.get(key) == value]
def first_n(self, n: int) -> List[Dict[str, Any]]:
"""Get first n records — useful for prompt sampling."""
return self.data[:n]
if __name__ == "__main__":
helper = DataHelper()
helper.add_item({"name": "Alice", "role": "user", "age": 30})
helper.add_item({"name": "Bob", "role": "assistant", "age": 25})
helper.add_item({"name": "Charlie", "role": "user", "age": 35})
print("All data (JSON):")
print(helper.to_json())
print("\nFiltered role=user:")
for rec in helper.filter_by("role", "user"):
print(rec)
print("\nFirst 2 records:")
print(helper.first_n(2))
Output
All data (JSON):
[
{
"name": "Alice",
"role": "user",
"age": 30
},
{
"name": "Bob",
"role": "assistant",
"age": 25
},
{
"name": "Charlie",
"role": "user",
"age": 35
}
]
Filtered role=user:
{'name': 'Alice', 'role': 'user', 'age': 30}
{'name': 'Charlie', 'role': 'user', 'age': 35}
First 2 records:
[{'name': 'Alice', 'role': 'user', 'age': 30}, {'name': 'Bob', 'role': 'assistant', 'age': 25}]
How it works
The DataHelper class wraps a simple list of dictionaries, making it easy to manage conversation records or training data for LLM prompts. add_item appends a record, to_json converts the list to a JSON string that many AI APIs accept as input. filter_by uses a list comprehension with .get to safely filter records by a key-value pair, avoiding KeyError when a key is missing. first_n slices the list to return a subset, which is handy for sampling prompts. The if __name__ == "__main__" block demonstrates usage, showing how the class integrates into a script.
Common mistakes
- Passing None to the constructor and accidentally sharing mutable default data
- Using `filter_by` without checking if the key exists in every record
- Forgetting to serialize with `json.dumps` before sending data to an LLM API
- Assuming `first_n` returns a copy rather than a reference when n equals list length
Variations
- Use `dataclasses` to define a `Record` class with typed fields and a `to_dict` method
- Add a `load_json` method to parse JSON strings back into the list
Real-world use cases
- Preparing and filtering chat history before sending it to an AI model for context.
- Storing and converting user feedback records into JSON for LLM fine-tuning datasets.
- Sampling a subset of logs to generate prompt examples for testing an AI assistant.
Sponsored
More from AI & LLM integration patterns
- Cache LLM Completions by Hashing the Prompt in Python easy
- Chain of Thought Prompting in Python: Step-by-Step Reasoning Demo easy
- Circuit Breaker Pattern in Python for LLM API Calls medium
- Cosine Similarity to Retrieve Top K Chunks in Python easy
- Demonstrate Prompt Injection Bypass in Python easy
- How to Accumulate Streamed Tokens into a Final String in Python easy
Keep learning
Related tutorials and quizzes for this topic.