In the article: Functions as Objects, we had seen that Python functions can accept another function as a parameter and can also return a function as its return value. Such functions that operate on another function are called Higher-Order Functions.
A decorator is a higher-order function that takes another function and extends its behaviour without explicitly modifying it.
Now that we know the definition, let's try to create a decorator, step-by-step 🪜. Let's take our square() from previous articles and try to extend its behaviour by logging its execution time (in seconds) - Let's call this the "Execution Time Logging" feature; ExTL in short 😉:
# Importing time module for calculating execution time
import time
def square(num):
return num * num
# Function to log execution time of square
def square_with_extl(num):
start_time = time.time()
result = square(num)
end_time = time.time()
print(f'Execution time: {end_time - start_time}s')
return result
print(square_with_extl(2))
# Execution time: 1.9073486328125e-06s
# 4
Well, this is a solution to our requirements. But, what if we wanted the same functionality for our add() too; Will you create another function: add_with_extl 😕? If we follow this approach, then we will end up creating a separate _with_extl function for all the functions that need the ExTL feature.
Instead, why don't we keep a single higher-order function: with_extl, which accepts the target function as a parameter, calls it to calculate the result, logs the execution time and then returns the result? That sounds like a better approach, right 😃? Let's implement and see 💻:
def add(num1, num2):
return num1 + num2
def with_extl(func, num):
start_time = time.time()
result = func(num)
end_time = time.time()
print(f'Execution time: {end_time - start_time}s')
return result
print(with_extl(square, 2))
# Execution time: 1.9073486328125e-06s
# 4
🤩 Nice! now with this with_extl(), we can add the ExTL functionality to any function - square(), add() etc. without having to create separate functions for each. But there is another problem 😒! Our with_extl() only accepts 1 parameter other than the target function, so how will we use it for add() which requires 2 parameters? Simple! we will keep the second argument as a variable-length argument. Also, let's update our log to include the function name as well.
def with_extl(func, *args):
start_time = time.time()
result = func(*args)
end_time = time.time()
print(f'{func.__name__}() executed in {end_time - start_time}s')
return result
print(with_extl(square, 2))
# square() executed in 1.9073486328125e-06s
# 4
print(with_extl(add, 2, 4))
# add() executed in 1.1920928955078125e-06s
# 6
😌 Excellent! Now our with_extl() has become much better and generic, such that we can use it for any function for which we need the ExTL. But, don't you see a shortcoming in our current implementation? 🤔
Our goal was to somehow extend the functionality of an existing function with ExTL, and then use the extended function just like the actual one. But as per our current implementation, every time, we need ExTL for a function, we are just making calls to the with_extl(), passing in the actual function along with its parameters. There is no actual "extending" happening here.🤨
If you recall the inner functions from Closure in Python, we could use it to wrap the ExTL logic and then return this inner function object from the with_extl() to solve our issue at hand. Let's try this out in code 💻:
def with_extl(func):
def inner(*args):
start_time = time.time()
result = func(*args)
end_time = time.time()
execution_time = end_time - start_time
print(f'{func.__name__}() executed in {execution_time}s')
return result
return inner
print(with_extl(square)(3))
# square() executed in 1.1920928955078125e-06s
# 9
square_with_extl = with_extl(square)
add = with_extl(add)
print(square_with_extl(2))
# square() executed in 4.76837158203125e-07s
# 4
print(add(2, 4))
# add() executed in 7.152557373046875e-07s
# 6
print(square(4)) # 16
Perfect! This is exactly what we wanted. The with_extl(), can now be called a decorator. 😇
Like the add() in our example, if we intend to use the function with the extended functionality alone, and not in its actual form, then there is another way of applying the decorator:💡
@with_extl
def add(num1, num2):
""" Returns the sum of two numbers """
return num1 + num2
print(add(2, 3))
# add() executed in 1.430511474609375e-06s
# 5
You will mostly see this format only when using decorators from standard/external libraries. Let's check one more thing. Let's print the value of some properties of the extended/decorated functions 💻:
print(add.__name__, square_with_extl.__name__) # inner inner
print(add.__doc__) # None
Ohh...We are losing the metadata while decorating 😖. Since we return the wrapper/inner function from the decorator function, the values that we see are of this function object, and not of our actual function. We can fix this by copying the metadata of our target function into this wrapper function using a decorator called @wraps of the functools standard library. With this information, let's refactor our with_extl decorator to be much more flexible and generic 👍:
from functools import wraps
def with_extl(func):
@wraps(func)
def wrapper(*args, **kwargs):
start_time = time.time()
result = func(*args, **kwargs)
end_time = time.time()
execution_time = end_time - start_time
print(f'{func.__name__}() executed in {execution_time}s')
return result
return wrapper
With this change, let's check the meta properties of our extended functions again 💻:
print(add.__name__, square_with_extl.__name__) # add square
print(add.__doc__) # Returns the sum of two numbers
😎 Much better! Our functions will now preserve their identity after decoration. Now that you know how to create a decorator and we already have the knowledge of using Function as Object, can you write the implementation of the @wraps decorator?
Use Cases
We already saw the use of the
@wrapsdecorator. Similarly, there are plenty of decorators available in Python's standard libraries -@contextlib.contextmanager,@functools.lru_cache,@asyncio.coroutineto name a few. We will look atsome of them in detail, in upcoming articles.Then there are decorators like
@staticmethod,@classmethod,@propertywhich are available globally and are used inside aClass. We will explore them in detail when we cover Class in Python. We will also look at how to create decorators using class and how to decorate a class.You can also create custom decorators or use them from external libraries for applications like debugging, caching return values and timing(like our with_extl). For example, the web framework:
Flaskand its ecosystem makes extensive use of decorators for registering plugins, rate limiting, validations, response marshalling etc.
Well, that’s all for this article. Thanks for reading! 🙏
I hope you liked this article and if you have any questions or suggestions, please drop them in the comments. Happy Coding.. 👋