Template Method Workflow Steps Base Class in Python

Define a reusable workflow skeleton in a base class and let subclasses fill in each step with the Template Method design pattern.

Medium Python 3.9+ Aug 9, 2026 System design patterns 13 views 0 copies

Python code

70 lines
Python 3.9+
from abc import ABC, abstractmethod


class DataPipeline(ABC):
    """Template Method pattern: defines a workflow skeleton."""

    def run(self):
        """Template method - defines the algorithm's structure."""
        result = {"extracted": False, "transformed": False, "loaded": False}
        raw_data = self._extract()
        result["extracted"] = True
        transformed = self._transform(raw_data)
        result["transformed"] = True
        self._load(transformed)
        result["loaded"] = True
        return result

    @abstractmethod
    def _extract(self):
        pass

    @abstractmethod
    def _transform(self, data):
        pass

    @abstractmethod
    def _load(self, data):
        pass


class CSVToDatabasePipeline(DataPipeline):
    """Concrete implementation for CSV to DB workflow."""

    def _extract(self):
        print("Extracting data from CSV...")
        return ["alice@example.com", "bob@example.com"]

    def _transform(self, data):
        print("Transforming emails...")
        return [email.strip().lower() for email in data]

    def _load(self, data):
        print(f"Loading {len(data)} rows into database...")
        for email in data:
            print(f"  - INSERT {email}")


class APIValidationPipeline(DataPipeline):
    """Concrete implementation for API validation workflow."""

    def _extract(self):
        print("Fetching API responses...")
        return {"status": "ok", "code": 200}

    def _transform(self, data):
        print("Validating response...")
        return data["status"] == "ok"

    def _load(self, data):
        print(f"Saving validation result: {data}")


if __name__ == "__main__":
    print("=== CSV Pipeline ===")
    csv_result = CSVToDatabasePipeline().run()
    print(f"Result: {csv_result}")

    print("\n=== API Pipeline ===")
    api_result = APIValidationPipeline().run()
    print(f"Result: {api_result}")

Output

stdout
=== CSV Pipeline ===
Extracting data from CSV...
Transforming emails...
Loading 2 rows into database...
  - INSERT alice@example.com
  - INSERT bob@example.com
Result: {'extracted': True, 'transformed': True, 'loaded': True}

=== API Pipeline ===
Fetching API responses...
Validating response...
Saving validation result: True
Result: {'extracted': True, 'transformed': True, 'loaded': True}

How it works

The DataPipeline base class defines a run() method that lays out the fixed algorithm: extract, transform, load. Subclasses implement only the abstract step methods, so the overall flow stays consistent while each step varies. ABC and @abstractmethod enforce that every concrete pipeline implements all required steps, preventing silent omissions. The template method centralizes shared logic like progress tracking and result reporting, so subclasses avoid duplicating orchestration code. This keeps the algorithm invariant while allowing steps to be customized per pipeline.

Common mistakes

  • Forgetting to call `super().run()` when overriding the template method in a subclass
  • Trying to instantiate the abstract base class directly instead of a concrete subclass
  • Adding new required steps without updating the abstract method list and all subclasses
  • Putting concrete step logic in the base class instead of leaving steps abstract

Variations

  1. Add a `hook()` method in the base class with a default no-op body that subclasses can optionally override
  2. Wrap each step in try/except blocks within the template method for centralized error handling

Real-world use cases

  • Defining a consistent ETL pipeline across different data sources and sinks in a data engineering project.
  • Standardizing API request lifecycle with auth, retry, and logging steps while allowing endpoint-specific parsing.
  • Creating a multi-stage model training workflow where preprocessing, training, and evaluation stages are customized per experiment.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from System design patterns

Related tutorials and quizzes for this topic.