RuntimeError: dictionary changed size during iteration in python

Question

I'm trying to change all the values of F to 1 and the values of M to 0 so I can create a dummy variable and then check the importance of Gender in my predicted results. I created a dictionary this way

Gender_dict = df_new.set_index("Student_ID") ["Gender"].to_dict() 

print (Gender_dict)

and got:

{366: 'F', 375: 'F', 381: 'F', 391: 'M', 399: 'M', 427: 'M', 429: 'M', 431: 'M', 435: 'M', 444: 'M', 452: 'F', 464: 'M', 472: 'F', 478: 'M', 484: 'F', 487: 'M', 495: 'M', 507: 'F', 1511: 'M', 1512: 'M', 1517: 'F', 1521: 'M', 1526: 'M', 1532: 'F', 1534: 'M', 1540: 'M', 1554: 'M', 1574: 'M', 1576: 'F', 1580: 'M', 1581: 'F', 1592: 'F', 1594: 'F', 1634: 'F', 1638: 'M', 1639: 'M', 1651: 'M', 1672: 'M', 2550: 'M', 7311: 'M', 7313: 'M', 7327: 'M', 7356: 'M', 7361: 'F', 7366: 'M', 7367: 'M', 7372: 'M', 7382: 'M', 7436: 'M', 7440: 'M', 7446: 'M', 8305: 'M', 8312: 'M', 8320: 'M', 8340: 'M', 8342: 'M', 8358: 'M', 8361: 'M', 8363: 'M', 8371: 'M', 8381: 'M', 8383: 'F', 8386: 'F', 8390: 'M', 8391: 'M', 8426: 'M', 8428: 'F', 8435: 'M', 8440: 'M', 8452: 'M', 8457: 'M', 9447: 'M', 9478: 'F', 9486: 'F', 9489: 'M', 9540: 'M', 9545: 'M', 9546: 'M'}

I thought this might work

for Student_ID, Gender in Gender_dict.items():
    if Gender == "F":
        Gender_dict[Gender] = "1"
    elif Gender == "M":
        Gender_dict[Gender] = "0"

print (Gender_dict)

But I get this error:

RuntimeError                              
Traceback (most recent call last)
<ipython-input-41-acce392dae9f> in <module>()
      5         #a1[color] = "Tulip"
      6 
----> 7 for Student_ID, Gender Gender_dict.items():

      8     if Gender == "F":
      9         Gender_dict[Gender] = "1"

RuntimeError: dictionary changed size during iteration

I tried adapting what I found to suit my purpose but can't get it to work. I also tried just about every .replace() and .apply() method I could find but nothing seems to work so I thought this would work.

Any help is greatly appreciated.

Answer 1

It's not totally clear what you're trying to achieve, but if you're doing one-hot encoding by hand, you just have typo in key name

for Student_ID, Gender in Gender_dict.items(): 
    if Gender == "F": 
        Gender_dict[Student_ID] = "1" 
    elif Gender == "M": 
        Gender_dict[Student_ID] = "0"

Answer 2

If you create a lookup dictionary:

gender_lookup = { 'F' : 1, 'M' : 0 }

Then you can update your other dictionary using a dictionary comprehension:

updated = { student_id : gender_lookup[gender] for student_id,gender in Gender_dict.items() } 

Answer 3

when iterating on a dictionary, it's perfectly all right to change values associated to an existing key.

What you cannot do is: add or remove keys.

You're accidentally doing this by using the value of the dictionary as the key, creating extra keys and generating the error message.

Generally, this kind of full dict update is better done using a dictionary comprehension, overriding the old dictionary:

Gender_dict = {Student_ID:"1" if Gender == "F" else "M" for Student_ID, Gender in Gender_dict.items()}

Answer 4

As the exception message suggests, you can't mutate a dict while you iterate over its items. You can iterate over a copy of the dict instead:

for Student_ID, Gender in list(Gender_dict.items()):