Confusions about python Iterators

Question

The more I practice Iterators, the more I get confused. I feel pretty confident in Objects and Classes (Only thing we have learned, Not learned inheritance) but Iterators and generators mess my head around. Any help is highly appreciated.

I have a some questions:

1) In the code below:

class main():
    def __init__(self):
        self.items=[1,2,3,4,5,6,7]
        self.index= 0

    def __iter__(self):
        return self 

    def __next__(self):
        self.index+=1

        return self.items[self.index]

a = main()

for i in a:
    print(i)  

1. We have two self here. One is in the init which is referring to the object 'a' and another one is returned by self. what is the real data type of self? is it of type main() or it is iterator?
2. Similar to the above question - when we give do next (self), what self are we giving to next (iterator or of type(a)) ?
3. If self after returned by __iter__ (also being used by next), is of the type iterator , how can be access self.index ?

2) In the code below I am trying to make iterate over specific thing such as keys or values or items in dictionary class. It is throwing an error ''iterator' object has no attribute 'index'. Why cant self.index access the instance variable index of dictionary class?

class Pair():
    def __init__(self, key ,value):
        self.key = key
        self.value = value

class Dictionary():
    def __init__(self):
        self.items =[]
        self.index = -1     ################## INDEX DEFINED HERE

    def __setitem__(self, key, value):
        for i in self.items:
            if i.key == key:
                i.value = value
                return
        self.items.append(Pair(key,value))

    def __keys__(self):
        return iterator(self, 'keys')

    def __values__(self):
        return iterator(self, 'values')

    def __items__(self):
        return iterator(self , 'items')

class iterator():
    def __init__(self, object, typo):
        self.typo = typo

    def __iter__(self):
        return self

    def __next__(self):
        if self.typo == 'keys': 
            self.index +=1  #################### ERROR
            if self.index >= len(self.items):
                raise StopIteration
            return self.items[self.index].keys

        ` # Similarly for keys and items as well`

collins = Dictionary()

collins['google'] = 'pixel'
collins['htc'] = 'M8'
collins['samsung'] = 'S9'


for i in collins.__keys__():
    print(i)

Answer 1

I have rewritten your code a bit with lots of comments to try and explain what is happening in example (1).

class MainClass():
    def __init__(self):
        # The value 'self' always refers to the object we are currently working
        # on. In this case, we are instantiating a new object of class
        # MainClass, so self refers to that new object.
        # self.items is an instance variable called items within the object
        # referred to as self.
        self.items = [1, 2, 3, 4, 5, 6, 7]
        # We do not want to declare self.index here. This is a slightly subtle
        # point. If we declare index here, then it will only be set when we first
        # create an object of class MainClass. We actually want self.index to be
        # set to zero each time we iterate over the object, so we should set it
        # to zero in the __iter__(self) method.
        # self.index = 0

    def __iter__(self):
        # This is an instance method, which operates on the current instance of
        # MainClass (an object of class MainClass). This method is called when
        # we start iteration on an object, so as stated above, we'll set
        # self.index to zero.
        self.index = 0
        return self

    def __next__(self):
        # This is also an instance method, which operates on the current
        # instance of MainClass.
        if self.index < len(self.items):
            self.index += 1
            return self.items[self.index - 1]
        else:
            # This is how we know when to stop iterating.
            raise StopIteration()


a = MainClass()

# a is now an object of class MainClass
# Because we have implemented __iter__ and __next__ methods in MainClass,
# objects of class MainClass are iterable, so a is also iterable.

# When we say "for i in a" this is like shorthand for  saying "a.__iter__()"
# and then "i = a.__next__()" until we raise
# a StopIterationException

# Here we are iterating over the result of a.__iter__() until a.__next__()
# raises a StopIterationException
for i in a:
    # Here we are printing the value returned by a.__next__()
    print(i)

I think it might help you to review this before you move on to (2) and double-check what you know about objects and classes. The first problem we can see in (2) is that you pass an object to your iterator class, but don't store it anywhere so you have no way to access it later. But you may find you have other ways you want to change it when you more fully understand all you have asked about in (1).

Answer 2

This answers only your first question, and might help you with question 2.

Citing from 'Fluent Python' (p. 420):

[...] Objects implementing an __iter__ method returning an iterator are iterable. [...]

That means, you could (in theory) do something like this:

class Main:
    def __init__(self):
        self.items = list(range(1, 8))
        self.length = len(self.items)

    def __iter__(self):
        return MainIterator(self)

Now, but how does the MainIterator class look like? The iterator just needs a __next__ dunder method to determine the next value it returns. An implementation could look like this:

class MainIterator:
    def __init__(self, iterable):
        self.iterable = iterable
        self.index = 0

    def __next__(self):
        if self.index >= self.iterable.length:
            raise StopIteration

        self.index += 1
        return self.iterable.items[self.index - 1]

What I am basically doing is creating a reference to the calling iterable and saving it in self.iterable . Now every time the __next__ dunder method is called, it returns an element of the array, until the iterator is exhausted. This is indicated by raising StopIteration .

You do not see such an implementation very often, as these two classes are often merged into a single class. I just wanted to demonstrate that it is possible to separate the two. The result is what @rbricheno already posted:

class Main:
    def __init__(self):
        self.items = list(range(1, 8))
        self.length = len(self.items)

    def __iter__(self):
        self.index = 0
        return self

    def __next__(self):
        if self.index >= self.length:
            raise StopIteration

        self.index += 1
        return self.items[self.index - 1]

The difference is that __init__ returns the instance itself, as the class itself is now iterable and iterator (remember: an iterator has the __next__ dunder method, and an iterable has a __iter__ dunder method that returns an iterator).

The last interesting bit is, when these dunder methods are called. Actually, when using the for in syntax, it is syntactic sugar for:

a = Main()

## recreating the for in loop

itr = a.__iter__()

while True:
    try:
        print(itr.__next__())
    except StopIteration:
        break

You initialize the iterator first, and __next__ returns a value until the iterator is exhausted.

EDIT:

You should really read my post again. It is NOT good practice to separate the iterator. It's just to demonstrate how they work internally. Also, please do not define your own dunder methods. That will break your code at some time. I have corrected your dict class below, but I iterate over the pair, not its components.

class Pair:

    def __init__(self, key, value):
        self.key = key
        self.value = value

    ## you need this to display your class in a meaningful way
    def __repr__(self):
        return f'{__class__.__name__}({self.key}, {self.value})'

class Dictionary:

    def __init__(self):
        self.items = []
        self.length = len(self.items)

    def add(self, objects):
        self.items.append(objects)
        self.length += 1

    def __iter__(self):
        self.index = 0
        return self

    def __next__(self):
        if self.index >= self.length:
            raise StopIteration

        self.index += 1
        return self.items[self.index - 1]

a = Dictionary()

a.add(Pair('up', 'above'))
a.add(Pair('down', 'below'))

for i in a:
    print(i.key)
    print(i.value)

Output on my machine:

up
above
down
below

Answer 3

Thats what I came up with:

class Pair():
    def __init__(self, key, value):
        self.key = key
        self.value = value


class dictionary():
    def __init__(self):
        self.items = []

    def add(self, objects):
        self.items.append(objects)

    def __keys__(self):
        return iterator(self, 'keys')

    def __values__(self):
        return iterator(self, 'values')

class iterator():
    def __init__(self, to_be_iterated , over_what):
        self.to_be_iterated = to_be_iterated
        self.over_what = over_what


    def __iter__(self):
        self.index = -1
        return self

    def __next__(self):
        self.index += 1
        if self.over_what == 'keys':
            try:
                    return self.to_be_iterated.items[self.index].key
            except Exception:
                raise StopIteration

        elif self.over_what == 'values':
            try:
                    return self.to_be_iterated.items[self.index].value
            except Exception:
                raise StopIteration


collins = dictionary()

collins.add(Pair('up', 'above'))
collins.add(Pair('down', 'below'))

for i in collins.__keys__():
    print(i)

for i in collins.__values__():
    print(i)