How to Vectorize a Function with a Pure Python Fallback
Create a decorator that calls a scalar function directly for a single value and routes list inputs to a pure-Python fallback for vectorized processing without NumPy.
Python code
34 linesimport math
def fallback_vectorize(func, fallback=None):
"""Vectorize a scalar function with a pure-Python fallback for lists."""
if fallback is None:
fallback = lambda x: [func(i) for i in x]
def wrapped(*args):
if len(args) == 1 and isinstance(args[0], (list, tuple)):
return fallback(args[0])
return func(*args)
return wrapped
@fallback_vectorize(math.sqrt, fallback=lambda xs: [math.sqrt(x) for x in xs])
def sqrt_or_error(x):
if x < 0:
raise ValueError("negative input not supported")
return math.sqrt(x)
if __name__ == "__main__":
# Single scalar call
print("Scalar:", sqrt_or_error(9))
# List call — routes to pure-Python fallback
print("Vector:", sqrt_or_error([4, 16, 25]))
# Mixed: list returns list, scalar returns scalar
data = [1, 4, 9]
result = sqrt_or_error(data)
print("Type:", type(result).__name__, "Values:", result)Output
Scalar: 3.0
Vector: [2.0, 4.0, 5.0]
Type: list Values: [1.0, 2.0, 3.0]How it works
The decorator fallback_vectorize wraps a scalar function so that when a single scalar argument is passed, it calls the original function directly. When a list or tuple is passed, it invokes the fallback function, which typically applies the scalar function to each element using a list comprehension. This provides a lightweight vectorization mechanism for environments where NumPy is not available or for simple data structures. The fallback is easily customizable, allowing the same wrapper to handle different types of collections or custom iteration logic.
Common mistakes
- Forgetting to handle the case where multiple arguments are passed, causing unexpected behavior.
- Assuming the fallback is only for lists, but tuples and other iterables might need support.
- Not preserving the original function's metadata (name, docstring) when using `functools.wraps`.
Variations
- Use `functools.wraps` to preserve the original function's name and docstring.
- Instead of a list comprehension, use `map` with the scalar function for lazy evaluation.
Real-world use cases
- Applying a scalar transformation to each element in a dataset when NumPy is not available in a constrained environment.
- Building a custom vectorization layer for a domain-specific library that needs to accept both scalars and arrays as input.
- Implementing a fallback path in a performance-critical function that can switch to a pure-Python implementation when optimized libraries are missing.
Sponsored
More from Concurrency & performance
- Benchmark list.append vs deque.append in Python medium
- Build a Python Performance Profiler That Generates Readable Reports medium
- Graceful Shutdown Executor Context Manager in Python medium
- How to Build a Producer-Consumer Pattern with asyncio.Queue in Python medium
- How to Cancel an asyncio Task with Graceful Cleanup in Python medium
- How to Convert Data in Parallel with ThreadPoolExecutor in Python easy
Keep learning
Related tutorials and quizzes for this topic.