We introduced generator functions as a way of writing functions that yield successive values while maintaining the function’s state in memory. We’ve seen how to use the yield statement to generate these values.
A generator function written this way does not allow any changes to the function’s variables to be passed in from the calling routine. In fact, there is a way of modifying a generator function’s behaviour from outside, by the use of the
send() method. Its use is a bit subtle, however, so we need a few examples to see how it works.
First, we write a generator function that generates a list of random integers, where the function’s arguments are the length of the list and the range of numbers from which the random integers are to be selected.
from random import * def genRandomList(lenList, rangeList): for i in range(10): ranList = [randint(2, 2 + rangeList) for i in range(lenList)] print('In genRandomList: ', ranList) vals = yield ranList if vals is not None: lenList = vals rangeList = vals
The function generates 10 random lists using list comprehension to generate each list on line 5.
There is a
yield statement on line 7, but here the
yield is given as the right operand of an assignment operator. We’ll see in a minute that when a generator yields a value to the caller, the caller has an opportunity to pass in some data to the generator. In this case, a two-element list is passed in and is assigned to
vals. After a check that
vals contains data on line 8, we assign its element  to be the new length of the list and its element  to be the new range of integers from which the random numbers are to be selected. Thus we have a way of changing the generator’s arguments after it has been called.
The send() method
The fact that we are assigning a
yield to a variable means that the generator expects something to be sent to it whenever a
yield is encountered. The way this is done is via the generator’s
send() method. To illustrate how it works, consider this code:
genList = genRandomList(4, 10) myList = next(genList) print(myList) myList = genList.send(myList[:2]) print(myList) myList = next(genList) print(myList) myList = next(genList) print(myList)
After initializing the generator, we call
next() to generate the first list. Then we call
send() with its argument being a list consisting of the first two elements of the
myList that was generated in the first step. This is the value that is passed into the generator’s code as the value of the yield statement, and is what gets assigned to the
vals variable on the left of the assignment operator. Since
vals is not
vals is assigned to be the new list length and
vals is the new range of integers. The call to
send() is followed with two more calls using
next(). Typical output is:
In genRandomList: [7, 9, 7, 8] [7, 9, 7, 8] In genRandomList: [10, 6, 9, 8, 8, 8, 11] [10, 6, 9, 8, 8, 8, 11] In genRandomList: [10, 10, 8, 8, 6, 8, 8] [10, 10, 8, 8, 6, 8, 8] In genRandomList: [10, 2, 10, 9, 10, 10, 2] [10, 2, 10, 9, 10, 10, 2]
The first list’s first two elements are 7 and 9, so the
send() call results in the next list being of length 7 with integers in the range 2 to 11. Note that the call to
send() acts like a call to
next() but with the extra feature of assigning a value to the yield statement. That is,
send() first assigns a value to
vals and then runs the generator code until the next
yield is encountered. In our example, this amounts to redefining the list’s parameters and then generating a new list.
The subsequent two calls to
next() don’t affect the list’s parameters, so they are both generated with a length of 7 and random numbers between 2 and 11. In fact, a call to
next() does pass something into the generator, but it’s the Python reserved word
None. This is the reason we included the check
if vals is not None on line 8 of the generator code above.
Now consider this code, in which we use a for loop to iterate through the generator:
genList = genRandomList(4, 10) for randList in genList: myList = next(genList) print(myList)
If you run this, you’ll find that only 5 lists are printed, rather than 10. The reason is that the
for statement is implicitly calling
next() on the generator object
genList at the start of each iteration of the loop. The value from this call to
next() is never used in the loop; rather we write an explicit call to
next() and print out the value from that. If we deleted this explicit call and just printed out the loop variable
randList, we would get all 10 lists. However, since we haven’t called
send() anywhere, the parameters of the list are never changed, since
next() always passes in
None to be assigned to vals.
Write some code that uses the
genRandomList() generator above to generate
n lists, where
n is the number of lists generated by
genRandomList(), and where each list uses the first two elements of the preceding list to define
rangeList. In the above code,
n is 10, but you should not have to know the value of
n to write this code.
The key is to use the
StopIteration exception to end the program cleanly. Suitable code is
genList = genRandomList(4, 10) myList = next(genList) while True: try: print(myList) myList = genList.send(myList[:2]) except StopIteration: break
The final call to
send() will generate the
Typical output (here with n = 13 in
[2, 11, 6, 5] [9, 11] [7, 4, 9, 5, 3, 2, 6, 7, 2] [2, 6, 5, 5, 3, 3, 4] [8, 5] [5, 2, 7, 2, 5, 3, 7, 2] [4, 4, 2, 4, 4] [3, 2, 6, 2] [2, 4, 2] [5, 5] [2, 6, 5, 6, 7] [7, 7] [3, 6, 2, 3, 2, 8, 4]
The throw() method
The above example relied on the generator running out of yields and thus raising a
StopIteration exception. We can force a generator to throw an exception (of any type, and at any time) by using the
throw() method. A simple example is the following, still using our
genList = genRandomList(4, 10) myList = next(genList) while True: print(myList) if len(myList) > 5: genList.throw(ValueError('List is too long')) myList = genList.send(myList[:2])
Running this code gives the typical output:
[2, 7, 3, 7] [8, 8] [8, 6, 2, 2, 5, 5, 2, 10] Traceback (most recent call last): File "D:\Documents\Programming\programmingpages\Python\Random list 01\Random list 01\Random_list_01.py", line 21, in <module> genList.throw(ValueError('List is too long')) File "D:\Documents\Programming\programmingpages\Python\Random list 01\Random list 01\Random_list_01.py", line 6, in genRandomList vals = yield ranList ValueError: List is too long
As soon as the list length exceeds 5, a
ValueError is thrown, with the message that we passed to it. Typically you would use a
try block to catch these exceptions and provide some clean way of handling them.
The close() method
close() method is really just a specialized version of
throw(), in that it always throws a
StopIteration. We could replace the line
genList.throw(ValueError('List is too long')) with
genList.close() in the above example, and a
StopIteration would be thrown when the first list with a length greater than 5 was generated.