Mock S3, GCS, and Azure storage with a Python abstract interface
Define an abstract Storage interface and implement a local, filesystem-backed mock so S3, GCS, and Azure code can be tested without cloud dependencies.
Python code
41 linesfrom abc import ABC, abstractmethod
from pathlib import Path
class Storage(ABC):
@abstractmethod
def put(self, name: str, data: bytes) -> None:
pass
@abstractmethod
def get(self, name: str) -> bytes:
pass
class LocalStorage(Storage):
def __init__(self, base_dir: str = "mock_store"):
self.base_dir = Path(base_dir)
self.base_dir.mkdir(exist_ok=True)
def put(self, name: str, data: bytes) -> None:
(self.base_dir / name).write_bytes(data)
def get(self, name: str) -> bytes:
return (self.base_dir / name).read_bytes()
def main() -> None:
storage = LocalStorage()
key = "example.txt"
content = b"Hello, storage!"
storage.put(key, content)
retrieved = storage.get(key)
print(f"Stored: {content!r}")
print(f"Retrieved: {retrieved!r}")
print(f"Match: {content == retrieved}")
if __name__ == "__main__":
main()
Output
Stored: b'Hello, storage!'
Retrieved: b'Hello, storage!'
Match: True
How it works
An ABC forces any cloud provider (S3, GCS, Azure) to implement put and get, keeping your business logic decoupled. The local mock writes bytes to files under a base directory, so tests run fast and offline. Swapping backends later is just a matter of choosing a different class that honors the same interface. This pattern is the core of cloud-agnostic storage code.
Common mistakes
- Forgetting to create the base directory before writing files.
- Mixing bytes and strings — cloud SDKs expect bytes, not str.
- Hardcoding a cloud SDK across your app instead of coding to the abstract interface.
Variations
- Add async methods for asyncio-based applications.
- Use fsspec as a unified file-system-like API for S3/GCS/Azure.
Real-world use cases
- Unit testing data pipelines locally without cloud costs or network access.
- Developing microservices that need interchangeable cloud storage during local development.
- Building integration tests that switch from mock to real providers in CI.
Sponsored
More from Cloud + Python
- Build a URL Shortener Client with Python medium
- Create a Cloud Storage Helper Class in Python easy
- Create a Data Helper Class for Beginners in Python easy
- Cross Account Role Chaining Mock Credentials in Python medium
- Exponential Backoff with Jitter for Cloud API Calls in Python medium
- Generate Mock CloudFormation Stack Events in Python easy
Keep learning
Related tutorials and quizzes for this topic.