pandas iterate over rows based on column values

Question

I want to calculate the temperature difference at the same time between to cities. The data structure looks as follows:

dic = {'city':['a','a','a','a','a','b','b','b','b','b'],'week':[1,2,3,4,5,3,4,5,6,7],'temp':[20,21,23,21,25,20,21,24,21,22]}
df = pd.DataFrame(dic)
df
+------+------+------+
| city | week | temp |
+------+------+------+
| a    |    1 |   20 |
| a    |    2 |   21 |
| a    |    3 |   23 |
| a    |    4 |   21 |
| a    |    5 |   25 |
| b    |    3 |   20 |
| b    |    4 |   21 |
| b    |    5 |   24 |
| b    |    6 |   21 |
| b    |    7 |   22 |
+------+------+------+

I would like to calculate the difference in temperature between city a and b at week 3, 4, and 5. The final data structure should look as follows:

+--------+-------+------+------+
| city_1 | city2 | week | diff |
+--------+-------+------+------+
| a      | b     |    3 |    3 |
| a      | b     |    4 |    0 |
| a      | b     |    5 |    1 |
+--------+-------+------+------+

Answer 1

I would pivot your data, drop the NA values, and do the subtraction directly. This way you can keep the source temperatures associated with each city.

result = (
    df.pivot(index='week', columns='city', values='temp')
      .dropna(how='any', axis='index')
      .assign(diff=lambda df: df['a'] - df['b'])  
)
print(result)

city     a     b  diff
week                  
3     23.0  20.0   3.0
4     21.0  21.0   0.0
5     25.0  24.0   1.0