Blog

We introduced generator functions as a way of writing functions that yield successive values while maintaining the function’s state in memory. We’ve seen how to use the yield statement to generate these values.

A generator function written this way does not allow any changes to the function’s variables to be passed in from the calling routine. In fact, there is a way of modifying a generator function’s behaviour from outside, by the use of the send() method. Its use is a bit subtle, however, so we need a few examples to see how it works.

First, we write a generator function that generates a list of random integers, where the function’s arguments are the length of the list and the range of numbers from which the random integers are to be selected.

from random import *

def genRandomList(lenList, rangeList):
    for i in range(10):
        ranList = [randint(2, 2 + rangeList) for i in range(lenList)]
        print('In genRandomList: ', ranList)
        vals = yield ranList
        if vals is not None:
            lenList = vals[0]
            rangeList = vals[1]

The function generates 10 random lists using list comprehension to generate each list on line 5.

There is a yield statement on line 7, but here the yield is given as the right operand of an assignment operator. We’ll see in a minute that when a generator yields a value to the caller, the caller has an opportunity to pass in some data to the generator. In this case, a two-element list is passed in and is assigned to vals. After a check that vals contains data on line 8, we assign its element [0] to be the new length of the list and its element [1] to be the new range of integers from which the random numbers are to be selected. Thus we have a way of changing the generator’s arguments after it has been called.

The send() method

The fact that we are assigning a yield to a variable means that the generator expects something to be sent to it whenever a yield is encountered. The way this is done is via the generator’s send() method. To illustrate how it works, consider this code:

genList = genRandomList(4, 10)
myList = next(genList)
print(myList)
myList = genList.send(myList[:2])
print(myList)
myList = next(genList)
print(myList)
myList = next(genList)
print(myList)

After initializing the generator, we call next() to generate the first list. Then we call send() with its argument being a list consisting of the first two elements of the myList that was generated in the first step. This is the value that is passed into the generator’s code as the value of the yield statement, and is what gets assigned to the vals variable on the left of the assignment operator. Since vals is not None, vals[0] is assigned to be the new list length and vals[1] is the new range of integers. The call to send() is followed with two more calls using next(). Typical output is:

In genRandomList:  [7, 9, 7, 8]
[7, 9, 7, 8]
In genRandomList:  [10, 6, 9, 8, 8, 8, 11]
[10, 6, 9, 8, 8, 8, 11]
In genRandomList:  [10, 10, 8, 8, 6, 8, 8]
[10, 10, 8, 8, 6, 8, 8]
In genRandomList:  [10, 2, 10, 9, 10, 10, 2]
[10, 2, 10, 9, 10, 10, 2]

The first list’s first two elements are 7 and 9, so the send() call results in the next list being of length 7 with integers in the range 2 to 11. Note that the call to send() acts like a call to next() but with the extra feature of assigning a value to the yield statement. That is, send() first assigns a value to vals and then runs the generator code until the next yield is encountered. In our example, this amounts to redefining the list’s parameters and then generating a new list.

The subsequent two calls to next() don’t affect the list’s parameters, so they are both generated with a length of 7 and random numbers between 2 and 11. In fact, a call to next() does pass something into the generator, but it’s the Python reserved word None. This is the reason we included the check if vals is not None on line 8 of the generator code above.

Now consider this code, in which we use a for loop to iterate through the generator:

genList = genRandomList(4, 10) 
for randList in genList:
    myList = next(genList)
    print(myList)

If you run this, you’ll find that only 5 lists are printed, rather than 10. The reason is that the for statement is implicitly calling next() on the generator object genList at the start of each iteration of the loop. The value from this call to next() is never used in the loop; rather we write an explicit call to next() and print out the value from that. If we deleted this explicit call and just printed out the loop variable randList, we would get all 10 lists. However, since we haven’t called send() anywhere, the parameters of the list are never changed, since next() always passes in None to be assigned to vals.

Exercise

Write some code that uses the genRandomList() generator above to generate n lists, where n is the number of lists generated by genRandomList(), and where each list uses the first two elements of the preceding list to define lenList and rangeList. In the above code, n is 10, but you should not have to know the value of n to write this code.

genList = genRandomList(4, 10)
myList = next(genList)
while True:
    try:
        print(myList)
        myList = genList.send(myList[:2])
    except StopIteration:
        break

[2, 11, 6, 5]
[9, 11]
[7, 4, 9, 5, 3, 2, 6, 7, 2]
[2, 6, 5, 5, 3, 3, 4]
[8, 5]
[5, 2, 7, 2, 5, 3, 7, 2]
[4, 4, 2, 4, 4]
[3, 2, 6, 2]
[2, 4, 2]
[5, 5]
[2, 6, 5, 6, 7]
[7, 7]
[3, 6, 2, 3, 2, 8, 4]

The throw() method

The above example relied on the generator running out of yields and thus raising a StopIteration exception. We can force a generator to throw an exception (of any type, and at any time) by using the throw() method. A simple example is the following, still using our genRandomList() generator.

genList = genRandomList(4, 10)
myList = next(genList)
while True:
    print(myList)
    if len(myList) > 5:
        genList.throw(ValueError('List is too long'))
    myList = genList.send(myList[:2])

Running this code gives the typical output:

[2, 7, 3, 7]
[8, 8]
[8, 6, 2, 2, 5, 5, 2, 10]
Traceback (most recent call last):
  File "D:\Documents\Programming\programmingpages\Python\Random list 01\Random list 01\Random_list_01.py", line 21, in <module>
    genList.throw(ValueError('List is too long'))
  File "D:\Documents\Programming\programmingpages\Python\Random list 01\Random list 01\Random_list_01.py", line 6, in genRandomList
    vals = yield ranList
ValueError: List is too long

As soon as the list length exceeds 5, a ValueError is thrown, with the message that we passed to it. Typically you would use a try block to catch these exceptions and provide some clean way of handling them.

The close() method

The close() method is really just a specialized version of throw(), in that it always throws a StopIteration. We could replace the line genList.throw(ValueError('List is too long')) with genList.close() in the above example, and a StopIteration would be thrown when the first list with a length greater than 5 was generated.