How to Serialize Cache Values with JSON and Pickle in Python

Serialize cache values using JSON for simple types or pickle for arbitrary objects, with robust error handling for unsupported types like mocks.

Medium Python 3.9+ Aug 9, 2026 Caching & Redis 12 views 0 copies

Python code

34 lines
Python 3.9+
import json
import pickle
from unittest.mock import Mock

def serialize(value, method="json"):
    """Serialize a cache value using JSON or pickle with type checking."""
    if method == "json":
        try:
            return json.dumps(value).encode("utf-8")
        except TypeError as e:
            raise ValueError(f"Cannot serialize {type(value).__name__} with JSON: {e}")
    elif method == "pickle":
        return pickle.dumps(value)
    else:
        raise ValueError(f"Unknown serialization method: {method}")

if __name__ == "__main__":
    # Test with basic types
    cache_data = {"user": "alice", "count": 42, "tags": ["a", "b"]}
    json_bytes = serialize(cache_data, "json")
    pickled_bytes = serialize(cache_data, "pickle")
    print(f"JSON: {json_bytes}")
    print(f"Pickle: {pickled_bytes[:50]}...")
    
    # Mock object that mimics a real cache value (e.g., from a test)
    mock_value = Mock()
    mock_value.data = {"status": "cached"}
    print(f"Mock repr: {mock_value!r}")
    
    # Note: JSON cannot handle Mock objects
    try:
        serialize(mock_value, "json")
    except ValueError as e:
        print(f"JSON mock error: {e}")

Output

stdout
JSON: b'{"user": "alice", "count": 42, "tags": ["a", "b"]}'
Pickle: b'\x80\x04\x95...'
Mock repr: <Mock id='140123456789012'>
JSON mock error: Cannot serialize Mock with JSON: Object of type Mock is not JSON serializable

How it works

JSON serialization works well for primitives, lists, and dictionaries but raises TypeError for custom objects. The try/except catches this and converts it to a more informative ValueError. Pickle handles any object but is unsafe to load from untrusted sources. Encoding JSON to UTF-8 bytes makes it suitable for storage in Redis or file caches. Type checking via type(value).__name__ helps debug serialization failures quickly.

Common mistakes

  • Forgetting to encode JSON strings to bytes for cache storage
  • Using pickle deserialization on untrusted data (security risk)
  • Not handling TypeError when JSON encounters non-serializable objects

Variations

  1. Use `msgpack.packb` for faster, compact serialization
  2. Use `orjson.dumps` for faster JSON with additional type support

Real-world use cases

  • Caching API responses in Redis with JSON-encoded bytes for fast retrieval.
  • Persisting complex Python objects across service restarts with pickle for internal state.
  • Testing cache layers with mock objects to verify serialization behavior before production use.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Caching & Redis

Related tutorials and quizzes for this topic.