Reference library

Data pipelines & processing

ETL-style flows, batch transforms, validation, and moving data between formats.

5 matches

Create Data Helper Functions in Python for Beginners

Build reusable Python helper functions to load, filter, sort, summarize, and save JSON data — a beginner-friendly starting point for small data pipelines.

json pipeline helpers

Python

import json
from pathlib import Path
from typing import Any, Dict, List


def load_json_file(filepath: str) -> Dict[str, Any]:
    """Load JSON data from a file."""
    with Path(filepath).open("r", encoding="utf-8") as file:
        return json.load(file)


def filter_by_key(
    data: List[Dict[str, Any]], key: str,…

14 0 Open

Data pipelines & processing easy

How to Build a Simple Data Pipeline in Python

A beginner-friendly data pipeline that loads JSON, filters records by a field value, and aggregates counts per category.

pipeline json aggregation

Python

import json
from pathlib import Path


def load_json(filepath: str | Path) -> list[dict]:
    """Load a JSON file containing a list of records."""
    with Path(filepath).open("r", encoding="utf-8") as f:
        return json.load(f)


def filter_records(records: list[dict], field: str, value) -> list[dict]:
    """Kee…

10 0 Open

Data pipelines & processing easy

How to Merge Multiple Data Sources in Python

A beginner-friendly helper that merges lists of dictionaries from multiple sources into one combined list using key filtering.

merge pipelines dicts

Python

import json

def merge_pipeline_data(*data_sources, keys=()):
    """Merge multiple data sources (list of dicts) into a single list of merged dicts.
    
    Args:
        *data_sources: One or more lists of dictionaries.
        keys: Tuple of keys to include from each source (empty means all keys).
    Returns:
    …

14 0 Open

Data pipelines & processing easy

How to Parse Data in Python: A Beginner's Helper

This helper parses a JSON payload, extracts user names, emails, and signup dates, then summarizes the results.

json parsing data-processing

Python

import json
from datetime import datetime
from typing import Dict, List


def parse_data(payload: str) -> Dict[str, List]:
    """Parse a JSON payload and extract useful fields."""
    raw = json.loads(payload)
    users = raw.get("users", [])

    parsed = {
        "names": [],
        "emails": [],
        "signup_…

15 0 Open

Data pipelines & processing easy

How to Process CSV Data in Python with a Data Helper

Build a beginner-friendly data helper in Python that loads a CSV file, filters rows by a condition, and summarizes numeric fields.

csv data-processing pathlib

Python

import csv
from pathlib import Path

DATA = [
    {"name": "Alice", "score": 88, "passed": True},
    {"name": "Bob", "score": 42, "passed": False},
    {"name": "Carol", "score": 95, "passed": True},
]


def load_csv(file_path: Path) -> list[dict]:
    with file_path.open(newline="", encoding="utf-8") as f:
        r…

13 0 Open

Browse by section

Each section groups closely related Python snippets.