How to Parse and Extract Nested Data in Python

Load JSON files with Path and recursively extract values by key from nested Python structures using modern typing and standard library.

Easy Python 3.9+ Aug 9, 2026 Modern tooling 12 views 0 copies

Python code

42 lines
Python 3.9+
import json
from pathlib import Path
from typing import Any, Dict, List, Union

def load_data(filepath: Union[str, Path]) -> Union[Dict[str, Any], List[Any]]:
    """Load JSON data from a file with modern Path handling."""
    path = Path(filepath)
    if not path.exists():
        raise FileNotFoundError(f"File not found: {path}")
    with path.open("r", encoding="utf-8") as f:
        return json.load(f)

def extract_values(data: Union[Dict[str, Any], List[Any]], key: str) -> List[Any]:
    """Recursively extract all values for a given key from nested structures."""
    results = []
    if isinstance(data, dict):
        for k, v in data.items():
            if k == key:
                results.append(v)
            results.extend(extract_values(v, key))
    elif isinstance(data, list):
        for item in data:
            results.extend(extract_values(item, key))
    return results

if __name__ == "__main__":
    sample_data = {
        "users": [
            {"name": "Alice", "age": 30},
            {"name": "Bob", "age": 25}
        ],
        "metadata": {"source": "demo"}
    }
    print("Extracted names:", extract_values(sample_data, "name"))
    print("Extracted ages:", extract_values(sample_data, "age"))

    # Demonstrate file loading with a small temp file
    temp_file = Path("sample.json")
    temp_file.write_text(json.dumps(sample_data), encoding="utf-8")
    loaded = load_data(temp_file)
    print("Loaded from file:", loaded["metadata"]["source"])
    temp_file.unlink()

Output

stdout
Extracted names: ['Alice', 'Bob']
Extracted ages: [30, 25]
Loaded from file: demo

How it works

The Path object provides a modern, cross-platform way to handle file paths without string concatenation. path.open() uses a context manager to ensure the file is closed automatically even if an error occurs. The recursive extract_values function walks through nested dicts and lists, collecting every value that matches the target key — this works for arbitrarily deep structures. Using Union[Dict[str, Any], List[Any]] type hints makes the function's contract clear to tools like mypy. The if __name__ == "__main__" guard keeps the demo code from running when the module is imported.

Common mistakes

  • Forgetting to use `encoding='utf-8'` when reading files, leading to UnicodeDecodeError on Windows.
  • Calling `extract_values` on the same list multiple times without a fresh `results` list, causing duplicate output.
  • Assuming all keys exist at the top level instead of using recursion for nested structures.
  • Not checking `path.exists()` before attempting to open a file, masking useful error messages.

Variations

  1. Use `json.loads(path.read_text(encoding='utf-8'))` as a one-liner alternative to `load_data`.
  2. Replace recursion with a stack-based iterative approach to avoid deep recursion limits on very large data.

Real-world use cases

  • Parsing API response JSON where the same field (e.g., 'id') appears at multiple nesting levels.
  • Loading configuration files that mix nested defaults with environment overrides and extracting a shared key.
  • Exploring a large dataset dump to find every occurrence of a specific field name for schema migration.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Modern tooling

Related tutorials and quizzes for this topic.