根据其他列创建新列，并在创建新列时根据上一行更新新列的行的值

Question

I am learning pandas in the past couple of months. I have a data frame like this:

       index  Random  id  diff  pct
0 2018-01-01      31   1     3    1
1 2018-01-02      11   1     2    2
2 2018-01-03      21   1     4    0
3 2018-01-04      23   2     1    0
4 2018-01-05      43   2     6    3
5 2018-01-06      42   2     1    1
6 2018-01-07      51   3     2    5
7 2018-01-08      47   3     2    0
8 2018-01-09      49   3     3    2
9 2018-01-10      22   3     1    3

What I want is to create a column recommend by 'Yes' and 'NO' by conditioning on other columns which I can do, but I also need to update the value of the 'Random' column on each row(or create a new column) with updating info for random column if the 'recommend' Column is Yes. For instance, The condition is if pct<diff, then 'recommand' column will be 'Yes' and 'Random'/'New_random' will be Random+diff, otherwise the 'recommand' column will be 'No' and 'Random'/'New_random' will be Random value of the previous row. FYI, we have to update 'Random'/'New_Random' column if 'recommand' is yes for that row and later rows for each id. The expected output should look like this

       index  Random  id  diff  pct recommend  Random_new
0 2018-01-01      31   1     3    1         Y          32
1 2018-01-02      31   1     2    2         N          32
2 2018-01-03      31   1     4    0         Y          36
3 2018-01-04      23   2     1    0         Y          24
4 2018-01-05      23   2     6    3         Y          27
5 2018-01-06      23   2     1    1         N          27
6 2018-01-07      51   3     2    5         N          51
7 2018-01-08      51   3     2    0         Y          53
8 2018-01-09      51   3     3    2         Y          56
9 2018-01-10      51   3     1    3         N          56

I have tried np.where which only create the column but don't update the row value for 'Random_new'. I feel like I need to create a for loop with if else condition, but could not do it so far. The condition as bullet points:

• If pct < diff 'Random_new'[i] = 'Random'[i]+'Diff'[i] else 'Random_new'[i]='Random_new'[i-1]
• With updating that row also update the later rows for 'Random_new'
• This needs to be for each id(probably using groupby) separately

Answer 1

First I'm not sure how you filled your values in your example but shouldn't the first Random_new be equal to 31+3=34 instead of 32?

Anyway you can first create you recommend column (boolean seems better adapted than Y/N) then create the Random_new with apply (only when recommend is True) and finally fill ( ffill ) the values when grouped by id :

df['recommend'] = df['pct'] < df['diff']
df['Random_new'] = df.apply(lambda x: x['Random'] + x['diff'] if x['recommend'] else None, axis=1)
df = df.groupby('id').ffill()

Output:

        index  Random  diff  pct  recommend  Random_new
0  2018-01-01      31     3    1       True        34.0
1  2018-01-02      11     2    2      False        34.0
2  2018-01-03      21     4    0       True        25.0
3  2018-01-04      23     1    0       True        24.0
4  2018-01-05      43     6    3       True        49.0
5  2018-01-06      42     1    1      False        49.0
6  2018-01-07      51     2    5      False         NaN
7  2018-01-08      47     2    0       True        49.0
8  2018-01-09      49     3    2       True        52.0
9  2018-01-10      22     1    3      False        52.0

Edit: if you wanna keep the id column replace the last line with:

df = pd.concat([df['id'], df.groupby('id').ffill()], axis=1)

(kwarg as_index=False doesn't help in this case)