How to Load, Save, and Split JSON Data in Python

Provides helper functions to load, save, and split JSON dictionary data for simple ML pipeline preprocessing.

Easy Python 3.9+ Aug 9, 2026 ML engineering pipelines 13 views 0 copies

Python code

42 lines
Python 3.9+
import json
from pathlib import Path


def load_json_data(file_path):
    """Load JSON data from a file, returning an empty dict if missing."""
    path = Path(file_path)
    if path.exists():
        with path.open("r", encoding="utf-8") as f:
            return json.load(f)
    return {}


def save_json_data(data, file_path):
    """Save data to a JSON file with pretty formatting."""
    path = Path(file_path)
    with path.open("w", encoding="utf-8") as f:
        json.dump(data, f, indent=2)


def split_data(data, train_ratio=0.8):
    """Split dictionary data into train and test subsets by ratio."""
    items = list(data.items())
    split_index = int(len(items) * train_ratio)
    train_items = items[:split_index]
    test_items = items[split_index:]
    return dict(train_items), dict(test_items)


if __name__ == "__main__":
    sample = {"user1": 25, "user2": 30, "user3": 22, "user4": 35, "user5": 28}
    save_json_data(sample, "temp_data.json")

    loaded = load_json_data("temp_data.json")
    train, test = split_data(loaded, train_ratio=0.6)

    print("Loaded data:", loaded)
    print("Train set:", train)
    print("Test set:", test)

    # Cleanup temp file
    Path("temp_data.json").unlink()

Output

stdout
Loaded data: {'user1': 25, 'user2': 30, 'user3': 22, 'user4': 35, 'user5': 28}
Train set: {'user1': 25, 'user2': 30, 'user3': 22}
Test set: {'user4': 35, 'user5': 28}

How it works

The load_json_data function checks if a file exists before reading, preventing exceptions for missing files. save_json_data uses json.dump with indent=2 for readable output. split_data slices the dictionary items into train and test subsets based on the given ratio, preserving order. The __main__ block demonstrates the full workflow with a sample dataset and cleans up the temporary file.

Common mistakes

  • Using `json.load` instead of `json.loads` for string data, or vice versa.
  • Assuming the file exists without checking, leading to FileNotFoundError.
  • Not specifying encoding='utf-8' can cause Unicode issues on some platforms.

Variations

  1. Use `pandas.read_json` and `train_test_split` for larger datasets.
  2. Add a seed parameter to `split_data` for reproducible shuffling.

Real-world use cases

  • Loading raw feature data from JSON files before splitting into training and validation sets in a batch ML job.
  • Persisting intermediate pipeline outputs (e.g., cleaned features) to JSON for reproducibility.
  • Splitting a small labeled dataset (stored as a dictionary) into train/test for quick model prototyping.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from ML engineering pipelines

Related tutorials and quizzes for this topic.