Mock S3, GCS, and Azure storage with a Python abstract interface

Define an abstract Storage interface and implement a local, filesystem-backed mock so S3, GCS, and Azure code can be tested without cloud dependencies.

Medium Python 3.9+ Aug 9, 2026 Cloud + Python 14 views 0 copies

Python code

41 lines
Python 3.9+
from abc import ABC, abstractmethod
from pathlib import Path


class Storage(ABC):
    @abstractmethod
    def put(self, name: str, data: bytes) -> None:
        pass

    @abstractmethod
    def get(self, name: str) -> bytes:
        pass


class LocalStorage(Storage):
    def __init__(self, base_dir: str = "mock_store"):
        self.base_dir = Path(base_dir)
        self.base_dir.mkdir(exist_ok=True)

    def put(self, name: str, data: bytes) -> None:
        (self.base_dir / name).write_bytes(data)

    def get(self, name: str) -> bytes:
        return (self.base_dir / name).read_bytes()


def main() -> None:
    storage = LocalStorage()
    key = "example.txt"
    content = b"Hello, storage!"

    storage.put(key, content)
    retrieved = storage.get(key)

    print(f"Stored: {content!r}")
    print(f"Retrieved: {retrieved!r}")
    print(f"Match: {content == retrieved}")


if __name__ == "__main__":
    main()

Output

stdout
Stored: b'Hello, storage!'
Retrieved: b'Hello, storage!'
Match: True

How it works

An ABC forces any cloud provider (S3, GCS, Azure) to implement put and get, keeping your business logic decoupled. The local mock writes bytes to files under a base directory, so tests run fast and offline. Swapping backends later is just a matter of choosing a different class that honors the same interface. This pattern is the core of cloud-agnostic storage code.

Common mistakes

  • Forgetting to create the base directory before writing files.
  • Mixing bytes and strings — cloud SDKs expect bytes, not str.
  • Hardcoding a cloud SDK across your app instead of coding to the abstract interface.

Variations

  1. Add async methods for asyncio-based applications.
  2. Use fsspec as a unified file-system-like API for S3/GCS/Azure.

Real-world use cases

  • Unit testing data pipelines locally without cloud costs or network access.
  • Developing microservices that need interchangeable cloud storage during local development.
  • Building integration tests that switch from mock to real providers in CI.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Cloud + Python

Related tutorials and quizzes for this topic.