8 ways to use functools with examples in Python
In this article, we will explore the functools standard library module and 8 cool things you can do with it.
1. Caching
You can use the @cache decorator (previously known as @lru_cache) as a “simple lightweight unbounded function cache”.
A typical example is calculating the Fibonacci sequence, where you cache intermediate results to significantly speed up the computation.
from functools import cache
@cache
def fibonacci(n:int) ->int:
if n <= 1:
return n
return fibonacci(n - 1) + fibonacci(n - 2)
def fibonacci2(n:int) ->int:
if n <= 1:
return n
return fibonacci2(n - 1) + fibonacci2(n - 2)
if __name__ == '__main__':
for i in range(40):
print(fibonacci(i))
for i in range(40):
print(fibonacci2(i))
Due to all the repetitive calculations, it takes 28.30 seconds. Using @cache, it only takes 0.02 seconds.
You can use @cached_property to perform the same operation on properties.
2. Write fewer comment methods
By using the @total_ordering decorator, you can write just one out of eq(), lt(), le(), gt(), or ge(), and it will automatically provide the other methods for you. It reduces the amount of code and increases automation.
According to the documentation, the trade-off is slower execution speed and more complex stack traces. Additionally, the decorator does not override methods already declared in the class or its superclasses.
The term “dunder” comes from “double underscore”. In Python, “dunder methods” are also known as “magic methods” or “special methods”. They are a set of predefined methods that have names starting and ending with double underscores (e.g., init, str).
3. Freezing Functions
partial() can be used to partially wrap existing functions, allowing you to set default values where they are not provided.
For example, if I want the print() function to always end with a comma instead of a newline character, I can use partial() as follows:
from functools import partial
print_no_newline = partial(print, end=', ')
# Normal print() behavior:
for _ in range(3): print('test')
test
test
test
# My new frozen print() one:
for _ in range(3): print_no_newline('test')
test, test, test,
Another example is to freeze the pow() built-in function to always square by fixing the exp parameter to 2:
from functools import partial
# Using partial with the built-in pow function
square = partial(pow, exp=2)
# Testing the new function
print(square(4)) # Outputs: 16
print(square(5)) # Outputs: 25
Another example:
from functools import partial
def power(base, exponent):
return base ** exponent
square = partial(power, exponent=2)
cube = partial(power, exponent=3)
print(square(4)) # 16
print(cube(4)) # 64
By using partial(), you can simplify repetitive calls, improve code clarity, and create reusable components with preset configurations.
There is also partialmethod(), which behaves like partial() but is intended to be used as a method definition rather than being called directly.
4. Using Generic Functions
With the introduction of PEP 443, Python has added support for “monomorphic generic functions”.
These functions allow you to define a set of functions (variants) for a main function, where each variant handles different types of arguments.
The @singledispatch decorator coordinates this behavior, allowing functions to change their behavior based on the type of the argument.
from functools import singledispatch
@singledispatch
def process(data):
"""Default behavior for unrecognized types."""
print(f"Received data: {data}")
@process.register(str)
def _(data):
"""Handle string objects."""
print(f"Processing a string: {data}")
@process.register(int)
def _(data):
"""Handle integer objects."""
print(f"Processing an integer: {data}")
@process.register(list)
def _(data):
"""Handle list objects."""
print(f"Processing a list of length: {len(data)}")
# Testing the generic function
process(42) # Outputs: Processing an integer: 42
process("hello") # Outputs: Processing a string: hello
process([1, 2, 3]) # Outputs: Processing a list of length: 3
process(2.5) # Outputs: Received data: 2.5
In the example above, when we call the dispatch function, it will invoke the corresponding registered function based on the type of the passed parameter.
For data types that do not have a registered function, the default behavior (defined in the main @singledispatch decorated function) will be used.
This design helps make the code more organized and clear, especially when a function needs to handle different data types in different ways.
5. Help write better decorators
When creating decorators, it is a good practice to use functools.wraps to preserve the metadata of the original function, such as its name and docstring. This ensures that the decorated function maintains its characteristics.
When writing decorators in Python, it is recommended to use functools.wraps() to avoid losing the docstring and other metadata of the decorated function:
from functools import wraps
def mydecorator(func):
@wraps(func)
def wrapped(*args, **kwargs):
result = func(*args, **kwargs)
return result
return wrapped
@mydecorator
def hello(name: str):
"""Print a salute message"""
print(f"Hello {name}")
# Thanks to functools, the encapsulated metadata is preserved
print(hello.__doc__) # 'Print a salute message'
print(hello.__annotations__) # {'name': <class 'str'>}
# If functools.wraps is not present, it will print
print(hello.__doc__) # None
print(hello.__annotations__) # {}
This way, the metadata of the decorator function is preserved, making it easier for developers to understand the purpose and usage of the function.
6. Summarize Data or Accumulate Transformations
functools.reduce(func, iterable) is a function that accumulates the results by applying a function to the iterable elements from left to right.
Note that reduce() has been moved to the functools module in Python 3, while in Python 2, reduce() is a built-in function.
This is useful in various scenarios where data needs to be summarized or transformed in an accumulating manner.
Here is an example where I use it to aggregate operations on a list of numbers:
from functools import reduce
import operator
numbers = list(range(1, 11)) # [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
print(operator.add(1, 2)) # 3
print(reduce(operator.add, numbers)) # 55
print(reduce(operator.sub, numbers)) # -53
print(reduce(operator.mul, numbers)) # 3628800
print(reduce(operator.truediv, numbers)) # 2.7557319223985893e-07
7. Using functools.timeout for function timeout
You can use functools.timeout to set the maximum execution time for a function. If the function does not complete within the specified time, a timeout error (TimeoutError) will be raised.
from functools import timeout
@timeout(5)
def slow_function():
# Some time-consuming operations
import time
time.sleep(10)
return "Function completed successfully"
try:
result = slow_function()
print(result)
except TimeoutError:
print("Function took too long to execute")
8. Create a singleton using functools.singleton (Python 3.11+)
Starting from Python 3.11, the functools module includes singleton, which is a decorator that ensures a class has only one instance.
from functools import singleton
@singleton
class MySingletonClass:
def __init__(self):
print("Creating instance")
obj1 = MySingletonClass()
obj2 = MySingletonClass()
print(obj1 is obj2) # Output: True
These examples are just a small part of the many functionalities in the functools module in Python. It is a powerful tool for functional programming that can simplify the design and maintenance of code.
Reference
https://www.jdon.com/70794.html