Reference library

Data pipelines & processing

ETL-style flows, batch transforms, validation, and moving data between formats.

4 matches

How to Convert Data Types in a Python Data Pipeline

Demonstrates a simple Python data pipeline that converts string values to proper types (bool, int, float, datetime) and outputs structured JSON.

data-pipeline type-conversion json

Python

import json
from datetime import datetime

def convert_value(value):
    """Convert string values to appropriate Python types."""
    if value.lower() == "true":
        return True
    if value.lower() == "false":
        return False
    if value.isdigit():
        return int(value)
    try:
        return float(val…

11 0 Open

Data pipelines & processing easy

How to Safely Coerce Strings to Numbers in Python

A safe conversion function that turns strings into integers or floats, returning a fallback value when conversion fails.

type-conversion robust-parsing data-cleaning

Python

import math

def to_number(value, fallback=None):
    """Safely coerce a string to int or float, returning fallback on failure."""
    if isinstance(value, (int, float)):
        return value
    try:
        # Try int first for clean whole numbers
        return int(value)
    except (ValueError, TypeError):
        …

12 0 Open

Data pipelines & processing easy

How to create a dated snapshot path for a dataset in Python

Generate a versioned directory path combining a base directory, dataset name, and today's date, ready for creating snapshots in data pipelines.

date pathlib datasets

Python

import datetime
import os
from pathlib import Path


def snapshot_path(base_dir: str, dataset_name: str) -> Path:
    """Return a dated snapshot path for a dataset under a base directory."""
    today = datetime.date.today().isoformat()
    return Path(base_dir) / dataset_name / today


if __name__ == "__main__":
    …

15 0 Open

Data pipelines & processing easy

Rollback dataset to previous snapshot pointer in Python

A SnapshotManager class stores timestamped data snapshots and rolls back to the most recent snapshot at or before a target time.

snapshots rollback datetime

Python

from datetime import datetime, timedelta


class SnapshotManager:
    def __init__(self):
        self.snapshots = {}  # timestamp -> data
        self.current_pointer = None

    def create_snapshot(self, data):
        timestamp = datetime.now()
        self.snapshots[timestamp] = data
        self.current_pointer =…

13 0 Open

Browse by section

Each section groups closely related Python snippets.