如何根据前一列填充 NaN 值

Question

I have an initial column with no missing data (A) but with repeated values. How do I fill the next column (B) with missing data so that it is filled and the column on the left always has the same value on the right? I would also like any other columns to remain the same (C)

For example, this is what I have

    A    B     C
1   1    20    4
2   2    NaN   8
3   3    NaN   2
4   2    30    9
5   3    40    1
6   1    NaN   3

And this is what I want

    A    B     C
1   1    20    4
2   2    30*   8
3   3    40*   2
4   2    30    9
5   3    40    1
6   1    20*   3

Asterisk on filled values.

This needs to be scalable with a very large dataframe.

Additionally, if I had a value on the left column that has more than one value on the right side on separate observations, how would I fill with the mean?

Answer 1

You can use groupby on 'A' and use first to find the first corresponding value in 'B' (it will not select NaN ).

import pandas as pd

df = pd.DataFrame({'A':[1,2,3,2,3,1], 
                   'B':[20, None, None, 30, 40, None], 
                   'C': [4,8,2,9,1,3]})

# find first 'B' value for each 'A'
lookup = df[['A', 'B']].groupby('A').first()['B']

# only use rows where 'B' is NaN
nan_mask = df['B'].isnull()

# replace NaN values in 'B' with lookup values
df['B'].loc[nan_mask] = df.loc[nan_mask].apply(lambda x: lookup[x['A']], axis=1)

print(df)

Which outputs:

   A     B  C
0  1  20.0  4
1  2  30.0  8
2  3  40.0  2
3  2  30.0  9
4  3  40.0  1
5  1  20.0  3

If there are many NaN values in 'B' you might want to exclude them before you use groupby .

import pandas as pd

df = pd.DataFrame({'A':[1,2,3,2,3,1], 
                   'B':[20, None, None, 30, 40, None], 
                   'C': [4,8,2,9,1,3]})

# Only use rows where 'B' is NaN
nan_mask = df['B'].isnull()

# Find first 'B' value for each 'A'
lookup = df[~nan_mask][['A', 'B']].groupby('A').first()['B']

df['B'].loc[nan_mask] = df.loc[nan_mask].apply(lambda x: lookup[x['A']], axis=1)

print(df)

Answer 2

You could do sort_values first then forward fill column B based on column A. The way to implement this will be:

import pandas as pd
import numpy as np

x = {'A':[1,2,3,2,3,1],
     'B':[20,np.nan,np.nan,30,40,np.nan],
     'C':[4,8,2,9,1,3]}

df = pd.DataFrame(x)

#sort_values first, then forward fill based on column B
#this will get the right values for you while maintaing
#the original order of the dataframe
df['B'] = df.sort_values(by=['A','B'])['B'].ffill()
print (df)

Output will be:

Original data:

   A     B  C
0  1  20.0  4
1  2   NaN  8
2  3   NaN  2
3  2  30.0  9
4  3  40.0  1
5  1   NaN  3

Updated data:

   A     B  C
0  1  20.0  4
1  2  30.0  8
2  3  40.0  2
3  2  30.0  9
4  3  40.0  1
5  1  20.0  3