Data pipelines & processing
ETL-style flows, batch transforms, validation, and moving data between formats.
Enrich a stream with reference data by key lookup in Python
Uses streamz to join each incoming record to a reference dictionary by name, adding department and level fields or defaults.
from streamz import Stream
reference = {"alice": {"dept": "eng", "level": 3}, "bob": {"dept": "sales", "level": 5}}
def enrich(record):
name = record.get("name")
ref = reference.get(name)
joined = dict(record)
if ref:
joined.update(ref)
else:
joined["dept"] = "unknown"
joi…How to perform a star schema join in Python
Denormalize mock fact and dimension tables by building lookup dicts and enriching each sales fact with customer, product, and date attributes.
from datetime import date
# Mock dimension tables
customers = [
{"customer_id": 1, "name": "Alice", "city": "New York"},
{"customer_id": 2, "name": "Bob", "city": "Los Angeles"},
{"customer_id": 3, "name": "Carol", "city": "Chicago"},
]
products = [
{"product_id": 101, "name": "Laptop", "category": "…Browse by section
Each section groups closely related Python snippets.
Data pipelines & processing — Python code examples
What you will find here
This page collects data pipelines & processing snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.