Pipeline stage compose functions left to right in Python
Compose multiple functions into a left-to-right pipeline so each stage receives the output of the previous one.
Python code
22 linesdef compose(*funcs):
"""Compose functions left to right: compose(f, g, h)(x) == h(g(f(x)))"""
def composed(arg):
result = arg
for func in funcs:
result = func(result)
return result
return composed
if __name__ == "__main__":
def add_one(x):
return x + 1
def double(x):
return x * 2
def square(x):
return x ** 2
pipeline = compose(add_one, double, square)
result = pipeline(3)
print(f"Result: {result}") # ((3 + 1) * 2) ** 2 = 64
Output
Result: 64
How it works
The compose function returns a new function that chains the input functions in the order they were passed. When called, it starts with the initial argument and passes it through each function sequentially, updating the result at every step. The left-to-right order matches the way data flows through a pipeline: compose(f, g, h)(x) is equivalent to h(g(f(x))). This differs from the traditional mathematical compose in many libraries, which applies functions right-to-left. The implementation uses a closure over funcs so the composed callable remembers its stages, and it gracefully handles a single stage or even an empty list by returning the input unchanged.
Common mistakes
- Confusing left-to-right composition with the right-to-left order used by functools.reduce or math-style compose
- Assuming the pipeline is applied in reverse order when reading `compose(f, g, h)`
- Forgetting that each function must accept exactly one argument since the pipeline passes a single value through
- Mutating shared state inside stage functions, which can break reusability of the composed pipeline
Variations
- Use `functools.reduce(lambda acc, f: lambda x: f(acc(x)), funcs, lambda x: x)` for a functional one-liner
- Create an async version with `await` between stages for I/O-heavy pipelines
Real-world use cases
- Chaining data transformation stages like parsing, cleaning, and normalizing records in an ETL job.
- Building a request processing pipeline where each middleware stage enriches or validates the payload.
- Composing image processing steps such as resize, filter, and watermark in a batch media workflow.
Sponsored
More from Data pipelines & processing
- Add a UUID Surrogate Key to Each Row in a CSV with Python easy
- Attach Source File Metadata to Records in Python easy
- Build a Python Utility That Detects Duplicate Records Across Multiple Excel Sheets medium
- Check Null Rate Threshold in PySpark DataFrame medium
- Count Records Processed per Category in Python easy
- Create Data Helper Functions in Python for Beginners easy
Keep learning
Related tutorials and quizzes for this topic.