Python Lesson 6
Lesson outline
Lambda functions
Errors and exception handling
Composing functions
Recursive functions
Iterators and Generators
Function decorators
Currying arguments
Exercises
Lambda functions
Python encourages a functional programming approach, it treats the functions as objects and facilitates the use of functions as arguments. If you have three different functions
In this case we can introduce the so called Python anonymous or lambda functions, to avoid defining functions f1
, f2
, and f3
. These are oneliners consisting of a single statement whose result is the value returned. They are defined using the lambda
keyword that implies the definition of an anonymous function. The syntax is lambda
/argumentlist/~: expression. These are called anonymous functions because, lacking the def
keyword, they have no name. We can use the previous example and introduce as arguments anonymous functions
The use of lambda functions is most often done together with the map
or filter
functions (Beware that in most, if not all cases, these constructions can be replaced by a list comprehension). Let's see how to combine anonymous functions with the map
function.
The map
function syntax is map(
ffunction ,
sequence )
, and it applies ffunction to each one of the sequence elements. The output of this function is an iterator (however, before Python 3, a list was returned). We can illustrate the use of map using the function that converts from degrees Farenheit to Kelvin defined in the previous lesson.
The use of a lambda function makes unnecessary to define the Fahren_2_Kelvin
function
The lambda
function can have several arguments and this can be combined with the possibility of applying map
to several lists.
Note that if the lists have unequal lengths, the resulting iterator ends at the final point of the shortest list.
The filter
function allows for a simple and elegant way of selecting which elements in a sequence evaluate to True
under a given conditional statement. The syntax is filter(
function , ~ /sequence/ ~)
. In the following example we filter the even values in the 20-th row of the Pascal triangle
The result of filter
is also an iterator.
Errors and exception handling
A possible tool to avoid errors in your programs, making them more foolproof, is assert
. The syntax of this command is assert (condition), "Warning message string"
. When the condition evaluates to True
the program continues, however if it is False
the warning message is print and the program stops with and AssertionError
message. The following line of code check whether the time
variable is positive or zero before the program continues running
This allows for an easy check of your program input to test if the values are sound.
Sometimes, specially when user input is involved, the input may be not what the program expects and you can make the program digest the input and not die miserably. Imagine you expect the user to provide a float or integer as time
value. Notice that when you read with input
a value it is recorded as a string.
If the user provides a non numerical value the code crashes with a ValueError: could not convert string to float. If we want to avoid this we can use a try/except
block
You can specify wich kind of exception are you trapping with the syntax except ValueError
. You can also trap several exception types including them in a tuple.
You can make use of this to keep asking for a value until it is of the correct type.
You can have also some code block run independently of the success or failure of the try
block using finally
or code that runs if the try
block is successful using else
.
If needed, you can stop your Python
script raising an exception e.g. ~Exception("Argument ", x, " is not a float.")
A Python
script can also be forced to end using the sys
library
Composing functions in Python
We can define the composition of two functions of a single argument as follows
In this way, we can combine the Fahren_2_Kelvin
and Try_Float
functions
If the function f
has several variables, this can be coped with using tuple references
If both functions f
and g
have several variables, then
For example, the following function computes the parametric dumbell curve
The dist
function computes the distance to the origin of a point in the plane
Therefore, we can compute the distance to the origin of the points in the dumbell curve
Recursive functions
A recursive function is a function that calls itself during its execution. This implies that, to avoid falling into pitfalls, the function ought to have a valid termination condition. A function that is often used to exemplify recursion is the factorial function n! = n·(n-1)·…·2·1. We can compute the factorial in a simple iterative way as follows (please, be aware that this is a simple implementation of the function and that is neither terribly efficient nor accurate)
This is a clear example where recursion can be used as n! = n·(n-1)! and the termination condition is 0! = 1. Therefore a recursive implementation of the factorial is
In this function the termination condition is always fullfilled if the n value is an integer. We can benchmark both functions using the %timeit
magic function
The recursive function is roughly twice slower than the non-recursive one. We can better understand this using the Padovan sequence calculation. The Padovan sequence is the sequence of integers defined by the values P(n) = P(n-2) + P(n-3) with the initial values P(0) = P(1) = P(2) = 1. Therefore P = 1, 1, 1, 2, 2, 3, 4, 5, 7,… (sequence A000931 in the OEIS). We can program the n-th term of this sequence in an iterative and recurrent way
In this case the recursive function is clearly less efficient than the iterative one. The reason of this difference is that when we use the recursive implementation we need to compute several times the same values. For example, if we are computing P(10) one needs P(8) and P(7), and if you check the different values of P(n) needed to get the final result it is obvious that some values are computed several times, imposing a significant computational burden. This can be improved using memoization, i.e. saving the already computed values for example in a dictionary.
The new implementation is 10⁶ times faster than the previous recursive one and even roughly 50 times faster than the iterative implementation. This will improve for larger argument values.
Iterators and iterables versus generators
The first thing to understand in this section is what is the difference between iterables and iterators. As we have already seen, we can iterate -for example with a for
loop- over different objects, e.g. a list, an ndarray, a tuple, or a set. All these objects are iterables. However, not all iterables are also iterators though the reverse is true, an iterator is always an iterable. The difference lies in the fact that iterators have an associated __next__
method that is run when the next()
function is applied to the iterator and they can be created from iterables using the iter()
function on them. Therefore, an iterator returns one piece of data at a time, when requested by the user.
When we use for
or sum
over an iterable, Python automatically transforms it into an iterator. We can also perform this transformation using iter()
and adding the __next__
method to the object.
You can check that calling next
once more over these objects once the sum is performed will raise an StopIteration
exception. You can also use the iter
function to test if an object is an iterable as it can be applied only to iterables
Another concept related to iterators is the concept of a generator. A generator is a type of function that helps creating iterators. We can see how can we transform lists or tuples or other iterables into iterators using iter()
. This is done -in a transparent way for the user- everytime a for
loop is run over a list or other iterables that are not iterators.
Doing the same with a hash results in iterating over the dict keys.
The standard way to build iterators in Python is making use of generators. A generator is very similar to a function, with the difference that it inludes yield
statements that is the statement that turns a function into a generator. The generator outputs a series of results instead of a single object as a regular function. The way to access the different results provided by the generator is by calling it repeteadly, as in a for
loop. Everytime the generator is invoked the code in the generator is run until a yield
statement is found and the corresponding output is returned. The execution stops here and proceeds once the generator is called again, until a new instance of yield
is found. The local variables still exist and retain their previous values, which is utterly different to what happens with functions. There may be several yield instances in the generator code, and if a return
statement is found the code will stop raising a StopIteration
exception. The return value of a generator is an iterator, and as the next
function is run for the generator it will provide, one-by-one, the expected output values. A simple generator that outputs the same state names that our previous list is
Once the last yield
statement has produced its ouput, a further next
call finishes raising a StopIteration
exception. Once you start retreiving values from a generator it cannot be reset, but you can creat a new one.
A simple example of a generator that implements a counter follows
In case the expression used in the generator is a simple one, you can use a generator expression which is quite similar to a list comprehension. The following example outputs sequentially the square root of odd numbers less than ten
We can implement the Padovan sequence in a generator that will provide the corresponding term of the sequence upon invocation
It is important to take care of including a termination condition in this case. We can do this in several ways. A possible option is to include this termination into the generator making use of a return
statement.
There is another possibility that consists of calling the original generator from another generator that implements the termination condition.
You can also send a value to a generator, making use of the send
method as in this simple example, making the generator act as a coroutine.
The variable value
takes the value sent through the send
method and it is None
in case no value is sent. Applying the send
method is equivalent to a next
instance and the corresponding yield
statement value is returned. A generator has to be started with a next
statement before the send
method is applied. We can make use of this method to program a generator that is a counter that can be modified at will
From Python 3.3 you can use the yield from
expr statement, where expr evaluates to an iterable. This allows removing unnecessary for
loops. For example, these two generators are completely equivalent
This allows for the combination of generators in an easy and direct way. Note that, as we remarked when noticing the difference between range
and arange
, a generator can provide substantial memory gains when facing intensive calculations and data reading.
Function decorators
A decorator is a callable object able to modify a function (or a class), and upon its it returns a modified version of the function (or class). The defines and returns a function, called the wrapper. This is a simple example of a decorator
In order to avoid having two different versions of the same function name foo as in our previous example, the usual syntax for decoration in Python is different. We can replace the statement foo = example_decorator(foo)
by @example_decorator
just in front of the decorated function.
Using this syntax we can use our decorator with any single-argument function but not third-party functions that have been imported from modules. In this case you should use the previous syntax.
The previous decorator can be easily extended to accept functions with any number of arguments using tuple references
A common use of decorators is to perform a test of the validity of the arguments. For example, the different implementations of the Padovan sequence we have introduced depend on a single positive integer argument. We can define a decorator that tests wether the argument is a natural number or not
Another example where a decorator can be of interest is if we want to keep track of how many times a function has been called. This is a general case, that can cope with several positional and keyword arguments
We can also avoid the use of the @decorator
syntax as follows
A very interesting decorator purpose it to perform memoization in a transparent way. Let's assume that we have a function, func, that is computationally costly and it may be worth to save the obtained results for each evaluation. This can be easily done through a hash, and including the hash in a decorator allows for a clean and transparent implementation. We define a decorator called memoize
We can combine two decorators and we will do this to check how this memoization decorator works in an example, combining it with the decorator farewell
defined above
In this case we first apply to f the farewell
decorator, followed by the memoize
decorator. Therefore, everytime the function f
is executed the ¡¡¡Hola!!!
message will be displayed. We can examine the memoization in action as follows
Notice how the order of the decorators is very relevant and changes the final output.
Currying functions
Currying -named after the mathematician Haskell Curry- is a functional desing pattern. It is the technique of converting a function that takes multiple arguments into a sequence of functions that each takes a single argument. Therefore, currying a function F = f(x,y,z)/ that takes three arguments, creates three functions such that /h = g(x), j=h(y) F = j(z) or F = g(x)(y)(z).
We provide an approach to currying using the partial
function from the functools
library that allows for binding the arguments of a function and the signature
function from the inspect
library.
We first introduce partial
. If we have a function that computed the distance of a point (x,y,z) given in Cartesian coordinates to the origin
Let's assume that we are limited to work in two dimensions, in the z = 2 plane. Using partial
, we can fix z = 2
and use an anonymous function
You can also include new default values for existing keyword arguments.
Let's assume that we have a function prdct(x,y,z) = xyz and we intend to curry it into cprod(x)(y)(z) = xyz. We can perform this using partial
as follows
This has built a chain of functions but it is not a very elegant way. Let's improve it using partial
in a recursive way as a decorator and the signature
function that provides a list of arguments of the function
We define and apply the decorator
Exercises
Exercise 6.1: Build a set of N=100 random coordinate values x, y, and z with values between -10 and 10. Using an anonymous function, select which of those points are at a distance less than a given limit dmin to the origin.
Exercise 6.2: Define a function that uses recursion to compute the n-th line of the Pascal triangle (1; 1, 1; 1, 2, 1; 1, 3, 3, 1;…).
Exercise 6.3: Define a generator that, using the Eratosthenes sieve (see Exercises Lesson 4), outputs a prime number each time it is called up for prime numbers below a given integer value.
Exercise 6.4: We have included memoization in the calculation of the Padovan sequence using a hash. Using a decorator make it transparent and also prepare a recursive implementation of the Tribonacci sequence, a third order sequence defined by T[0] = 0, T[1] = T[2] = 1 and T[n] = T[n-1] + T[n-2] + T[n-3] for n>1. Compare the speed of the implementations with and without the memoization decorator.
Exercise 6.5: Use as a starting point the password generating function defined in Exercise 4.2. Add an argument that fix the generated parameter length. Then prepare a decorator that would (1) ensure that the argument is an integer or can be transformed into an integer; (2) check that the integer argument value is larger than or equal than six; and (3) perform the same task as in Exercise 4.2, check that there are at least two digits, two uppercase, and two lowcase characters and return the compliant password and the number of times the function has been run to get this password.
Last updated