Why does appending to a list break my for loop? (Python)

Question

for family in city.familyList:
    for member in family.membersList:
        if member.sex == "male":
            if member.status == "eligible":
                print "There were", len(family.membersList), "members of family", family.familyName
                family.membersList.remove(member)
                print "Now there are", len(family.membersList), "members of family", family.familyName
                member.status = "needs a wife"
                print member.firstName, member.status
                print "There were", len(city.familyList), "families"
                city.familyList.append(Family(member.lastName, member))
                print "Now there are", len(city.familyList), "families"

With this code I am attempting to iterate through a list of family members, locate males over 18, remove them from their family, and start their own family. If I comment out the append at the end of the loop it works fine (without increasing the number of families, of course). This is what it looks like if I perform the append:

Ticking Disan
There were 5 members of family Evnes
Now there are 4 members of family Evnes
Gregor needs a wife
There were 6 families
Now there are 7 families
There were 7 members of family Bhworth
Now there are 6 members of family Bhworth
Ewan needs a wife
There were 7 families
Now there are 8 families

debugger.run(setup['file'], None, None)
  File "C:\Users\Mark\Desktop\eclipse-SDK-3.7.2-win32\eclipse\plugins\org.python.pydev.debug_2.5.0.2012040618\pysrc\pydevd.py", line 1060, in run

pydev_imports.execfile(file, globals, locals) #execute the script
  File "C:\Users\Mark\workspace\WorldPop\WorldPop.py", line 610, in <module>
    main()
  File "C:\Users\Mark\workspace\WorldPop\WorldPop.py", line 74, in main
    done = menu()
  File "C:\Users\Mark\workspace\WorldPop\WorldPop.py", line 77, in menu
    genWorld()
  File "C:\Users\Mark\workspace\WorldPop\WorldPop.py", line 116, in genWorld
    dispWorld(oneWorld)
  File "C:\Users\Mark\workspace\WorldPop\WorldPop.py", line 135, in dispWorld
    displayTick(world)
  File "C:\Users\Mark\workspace\WorldPop\WorldPop.py", line 317, in displayTick
    calcMarriage(city)
  File "C:\Users\Mark\workspace\WorldPop\WorldPop.py", line 359, in calcMarriage
    for member in family.membersList:
TypeError: iteration over non-sequence

I realize that the problem comes when the for loop cycles back to the beginning to search through a new memberList, I just don't see why performing the append is breaking the loop. Any insight is greatly appreciated.

Answer 1

Your new Family instance that you're appending has a membersList attribute, but it does not support iteration (for/in). That's why you receive this exception.

Even after fixing that, you are setting yourself up for surprises by modifying the sequences while iterating over them. Try this:

for family in city.familyList[:]:
    for member in family.membersList[:]:
        # ...

The [:] slice syntax creates a copy of each list as it exists at the start of the loop. Modifications made to the original lists will not affect the copy, so there won't be any surprises during loop execution.

If you really need to include new items found/created during the for loop in the for loop, I recommend iterating over a Queue.Queue object and inserting each object to explore into that queue. When there's nothing new to explore the queue will be empty and the loop will stop.

Answer 2

In general it's a really bad idea to modify the collection you're iterating through (you're both removing from family.membersList and adding to city.familyList ).

Depending on the data structure that you're iterating over, and on the iteration algorithm, changing the iterated collection might cause items to be either skipped or seen more than once. I'd assume this is unsafe most of the time, unless the docs explicitly say otherwise.