Reference library

Data pipelines & processing

ETL-style flows, batch transforms, validation, and moving data between formats.

4 matches

ETL in Python: Extract CSV, Transform Dicts, Load JSON

Build a simple ETL pipeline that reads a CSV, normalizes keys and converts price to float, then writes structured JSON.

import csv
import json
from pathlib import Path

def etl_csv_to_json(csv_path: str, json_path: str) -> None:
    """Extract CSV, transform rows to dicts, load to JSON."""
    with open(csv_path, mode='r', newline='', encoding='utf-8') as f:
        reader = csv.DictReader(f)
        records = list(reader)

    # Trans…

12 0 Open

Data pipelines & processing medium

Extract Schema.org Structured Data from Any Website in Python

A Python tool that fetches a webpage and extracts all JSON-LD structured data (Schema.org) embedded in <script> tags with type="application/ld+json".

web-scraping structured-data schema-org

Python

import requests
from bs4 import BeautifulSoup
import json

def extract_schema_org(url):
    """Extract structured data (Schema.org) from a website."""
    try:
        response = requests.get(url, timeout=10)
        response.raise_for_status()
    except requests.exceptions.RequestException as e:
        return {"err…

50 0 Open

Data pipelines & processing easy

How to Convert Data Types in a Python Data Pipeline

Demonstrates a simple Python data pipeline that converts string values to proper types (bool, int, float, datetime) and outputs structured JSON.

data-pipeline type-conversion json

Python

import json
from datetime import datetime

def convert_value(value):
    """Convert string values to appropriate Python types."""
    if value.lower() == "true":
        return True
    if value.lower() == "false":
        return False
    if value.isdigit():
        return int(value)
    try:
        return float(val…

12 0 Open

Data pipelines & processing easy

How to Count JSON Records in Python

Read a JSON file and count the number of top-level records, handling both list and dictionary structures.

json counting file-reading

Python

import json
from pathlib import Path

def count_records(json_file):
    """Count top-level records in a JSON file."""
    with open(json_file, "r") as f:
        data = json.load(f)
    
    # Handle both list of records and dict of records
    if isinstance(data, list):
        return len(data)
    elif isinstance(da…

13 0 Open

Browse by section

Each section groups closely related Python snippets.