python itertools: Using cycle with islice

Question

Question:

I have the code below. I want to know why does it not matter whether or not I include the line with the comment in the code below.

#!/usr/bin/env python

from itertools import *
import time
cc = cycle([ iter([1,2,3]), iter([4]) , iter([5,6]) ] )
p = 3
while p:
    try:
        for k in cc:
            print k.next()
    except StopIteration:
        p = p - 1
        cc = cycle(islice(cc,  p))  # this does not matter

Output:

1
4
5
2
6
3

Also checkout roundrobin recipe at

http://docs.python.org/2.7/library/itertools.html

This code shows that islice is impacting cc

#!/usr/bin/env python

from itertools import *
import time
cc = cycle([ iter([1,2,3]), iter([4]) , iter([5,6]) ] )
p = 3
while p:
    try:
        for k in cc:
            print k,k.next()
    except StopIteration:
            print "stop iter"
            p = p - 1
            cc = cycle(islice(cc,  p)) 

Output

<listiterator object at 0x7f32bc50cfd0> 1
<listiterator object at 0x7f32bc518050> 4
<listiterator object at 0x7f32bc518090> 5
<listiterator object at 0x7f32bc50cfd0> 2
<listiterator object at 0x7f32bc518050> stop iter
<listiterator object at 0x7f32bc518090> 6
<listiterator object at 0x7f32bc50cfd0> 3
<listiterator object at 0x7f32bc518090> stop iter
<listiterator object at 0x7f32bc50cfd0> stop iter

Answer 1

Short course: the behaviors with and without rebinding cc aren't generally the same, but the outputs happen to be the same for the specific input you used.

Long course: let's call your three iterators A, B and C.

Without the cc rebinding: A produces 1, B produces 4, C produces 5, A produces 2, B raises StopIteration . p drops to 2. Then C produces 6, A produces 3, and B raises StopIteration again . p drops to 1. C raises StopIteration . p drops to 0, and the loop exits.

With the cc rebinding: A produces 1, B produces 4, C produces 5, A produces 2, B raises StopIteration . p drops to 2. All the same so far. The purpose of the rebinding in the round-robin algorithm is to remove exhausted iterators. It so happens that in this specific example it makes no difference to the outcome. With the rebinding islice(cc, 2) takes "the next two things" from cc , which are, in order, C and A. B is no longer there. Then C and A are put in a new cycle .

Then C produces 6, A produces 3, and C raises StopIteration . p drops to 1. The cc rebinding gets rid of C (the exhausted iterator), leaving just A in a new cycle . The loop goes around, and A raises StopIteration . p drops to 0, and we're done.

Of course removing exhausted iterators is crucial to making round-robin work correctly in general. But, as you've shown, there are specific cases in which that doesn't matter :-)

Simple example

A simple case where rebinding cc makes a huge difference:

cc = cycle([iter([1,2,3,4,5]), iter([])])
p = 2

With rebinding, we get all 5 values. Without rebinding, we only get 1 and 2.

Answer 2

well... it looks like it's doing what is expected here.

So, normally, cycle works like this:

cycle([1,2,3,4,5]) -> [1,2,3,4,5,1,2,3,4,5,1,2,3,4,5]

It's going to store off values until it gets a StopIterator, and then it's going to start returning values from its saved list. In this case, that's going to be [iter(a), iter(b), iter(c)] (where iter(x) is a listiterator object itself, across whatever is inside). So chain is actually returning something like this:

[iter(a), iter(b), iter(c), iter(a), iter(b), iter(c), iter(a), iter(b), iter(c), ...]

What looks looks like when you run it:

[1, 4, 5, 2, StopIterator]

But that's the value of k.next(), not k. the cc itself is not returning the StopIterator, the object it returns is.

Now, you call islice(cc,2) . This should return the sequence [iter(c), iter(a)] , as those are the next two items in the sequence. Again, you want cc to cycle them, but now you should get

[iter(c), iter(a), iter(c), iter(a), iter(c), iter(a), ...]

You won't notice much of a difference yet, but that's because your slice is less than the length of the original. In other words, the original would have gone [iter(c), iter(a), iter(b), iter(c), iter(a), ...]

but you have to go farther than two items to see that difference, and...

You start pulling items off and you get

[6, 3, StopIterator]

Only two items, so they are the same two items you'd get without the islice.

Now, when you do the islice(cc, 2) , you get the next two items which are [iter(a), iter(c)] . Both of these are exhausted, so you get

[StopIterator]

and your exact sequence.

I don't know what you are expecting, but this works exactly as i would expect it to.