How to Serialize Cache Values with JSON and Pickle in Python
Serialize cache values using JSON for simple types or pickle for arbitrary objects, with robust error handling for unsupported types like mocks.
Python code
34 linesimport json
import pickle
from unittest.mock import Mock
def serialize(value, method="json"):
"""Serialize a cache value using JSON or pickle with type checking."""
if method == "json":
try:
return json.dumps(value).encode("utf-8")
except TypeError as e:
raise ValueError(f"Cannot serialize {type(value).__name__} with JSON: {e}")
elif method == "pickle":
return pickle.dumps(value)
else:
raise ValueError(f"Unknown serialization method: {method}")
if __name__ == "__main__":
# Test with basic types
cache_data = {"user": "alice", "count": 42, "tags": ["a", "b"]}
json_bytes = serialize(cache_data, "json")
pickled_bytes = serialize(cache_data, "pickle")
print(f"JSON: {json_bytes}")
print(f"Pickle: {pickled_bytes[:50]}...")
# Mock object that mimics a real cache value (e.g., from a test)
mock_value = Mock()
mock_value.data = {"status": "cached"}
print(f"Mock repr: {mock_value!r}")
# Note: JSON cannot handle Mock objects
try:
serialize(mock_value, "json")
except ValueError as e:
print(f"JSON mock error: {e}")
Output
JSON: b'{"user": "alice", "count": 42, "tags": ["a", "b"]}'
Pickle: b'\x80\x04\x95...'
Mock repr: <Mock id='140123456789012'>
JSON mock error: Cannot serialize Mock with JSON: Object of type Mock is not JSON serializable
How it works
JSON serialization works well for primitives, lists, and dictionaries but raises TypeError for custom objects. The try/except catches this and converts it to a more informative ValueError. Pickle handles any object but is unsafe to load from untrusted sources. Encoding JSON to UTF-8 bytes makes it suitable for storage in Redis or file caches. Type checking via type(value).__name__ helps debug serialization failures quickly.
Common mistakes
- Forgetting to encode JSON strings to bytes for cache storage
- Using pickle deserialization on untrusted data (security risk)
- Not handling TypeError when JSON encounters non-serializable objects
Variations
- Use `msgpack.packb` for faster, compact serialization
- Use `orjson.dumps` for faster JSON with additional type support
Real-world use cases
- Caching API responses in Redis with JSON-encoded bytes for fast retrieval.
- Persisting complex Python objects across service restarts with pickle for internal state.
- Testing cache layers with mock objects to verify serialization behavior before production use.
Sponsored
More from Caching & Redis
- Cache Asides in Python with a Read-Through Loader easy
- Cache Data in Redis with Python easy
- Cache Penetration Null Object Mock in Python medium
- Cache Stampede Prevention with SingleFlight in Python medium
- Cache Warming with Python: Preload Hot Keys easy
- Coalescing duplicate in-flight requests: one shared result for concurrent callers hard
Keep learning
Related tutorials and quizzes for this topic.