Cache persist MEMORY_ONLY mock in Python
Mock a MEMORY_ONLY persistence cache in Python with an LRU eviction policy and optional persistence flag.
Python code
50 linesimport time
class LRUCache:
def __init__(self, capacity, persistence="MEMORY_ONLY"):
self.capacity = capacity
self.persistence = persistence
self.cache = {}
self.access_order = []
self.hits = 0
self.misses = 0
def get(self, key):
if key in self.cache:
self.hits += 1
self.access_order.remove(key)
self.access_order.append(key)
return self.cache[key]
self.misses += 1
return None
def put(self, key, value):
if key in self.cache:
self.access_order.remove(key)
self.access_order.append(key)
self.cache[key] = value
return
if len(self.cache) >= self.capacity:
evicted = self.access_order.pop(0)
del self.cache[evicted]
print(f"Evicted: {evicted}")
self.cache[key] = value
self.access_order.append(key)
def stats(self):
return {
"persistence": self.persistence,
"size": len(self.cache),
"hits": self.hits,
"misses": self.misses,
"cache": dict(self.cache),
}
if __name__ == "__main__":
cache = LRUCache(capacity=3, persistence="MEMORY_ONLY")
cache.put("alice", 25)
cache.put("bob", 30)
cache.put("carol", 35)
cache.get("alice")
cache.put("dave", 40)
print(cache.stats())
Output
Evicted: bob
{'persistence': 'MEMORY_ONLY', 'size': 3, 'hits': 1, 'misses': 0, 'cache': {'alice': 25, 'carol': 35, 'dave': 40}}
How it works
The LRUCache class simulates an in-memory cache with a configurable persistence level, defaulting to MEMORY_ONLY. The put method evicts the least recently used item when capacity is reached, updating an access_order list. The get method records hits and misses while refreshing recency. The stats method returns a dictionary with cache size, hit/miss counters, and the current contents, mirroring Spark's persistence level concept.
Common mistakes
- Forgetting to move existing keys to the end of the access order on `put` to maintain LRU order
- Not handling duplicate keys in `access_order` when removing them
- Assuming `cache` is thread-safe when accessing from multiple threads without locks
Variations
- Use `OrderedDict` with `move_to_end` for a more concise LRU implementation
- Add a TTL expiry time to entries for time-based cache eviction
Real-world use cases
- Simulating Spark's MEMORY_ONLY persistence in unit tests for data pipelines before tuning real jobs.
- Implementing a lightweight in-process cache for frontend services that must avoid disk I/O latency.
- Building a prototype for a caching layer in a recommendation engine to test eviction behavior.
Sponsored
More from Big data & Spark
- Accumulators Global Counter Mock in Python medium
- Approximate Distinct Count in Python with HyperLogLog medium
- Bloom Filter Join Mock in Python medium
- Compaction Small Files Mock in Python easy
- Delta Lake ACID Transaction Log Mock in Python medium
- How to Broadcast a Small Lookup Table in Python easy
Keep learning
Related tutorials and quizzes for this topic.