同时填充pandas数据帧中相关列中的缺失值

Question

I have a dataframe with two columns State and Code, with missing values in each.

import pandas as pd

df = pd.DataFrame([['Alabama', 'AL'], ['Alaska', 'AK'], ['Arizona', 'AZ'], ['Arkansas', 'AR'], ['Iowa','IA'],['Hawaii','HI'], ['Idaho', 'ID'], ['Alabama', ''], ['', 'IA'], ['Alaska',''], ['', 'AZ']], columns=['State', 'Code'])

Missing values

    State   Code
7   Alabama     
8             IA
9   Alaska  
10            AZ

What I've tried

state_code_dict = {
    'Alabama': 'AL',
    'Alaska': 'AK',
    'Arizona': 'AZ',
    'Arkansas': 'AR',
    'Iowa':'IA',
    'Hawaii':'HI',
    'Idaho': 'ID',    
}

def state_code(x):
    if (x['Code'] == ''):
        return state_code_dict[x['State']]
    else:
        return x['Code']

df['Code'] = df.apply(lambda x: state_code(x), axis=1)

This sets the missing values in Code. I need to update this function for setting State as well. I'm looking to simplify this.

Required output

    State   Code
7   Alabama   AL
8   Iowa      IA
9   Alaska    AK
10  Arizona   AZ

Answer 1

IIUC, you can use map to first map codes and then states, using boolean masking to just assign values when you have empty values

mask = df.Code == ''
df.loc[mask, 'Code'] = df[mask].State.map(state_code_dict)

mask = df.State == ''
df.loc[mask, 'State'] = df[mask].Code.map({v:k for k,v in state_code_dict.items()})

    State   Code
0   Alabama AL
1   Alaska  AK
2   Arizona AZ
3   Arkansas    AR
4   Iowa    IA
5   Hawaii  HI
6   Idaho   ID
7   Alabama AL
8   Iowa    IA
9   Alaska  AK
10  Arizona AZ

Answer 2

You can replace blank strings with np.nan and then use fillna with pd.Series.map . Similar idea to @RafaelC but implemented differently.

code_state_dict = {v: k for k, v in state_code_dict.items()}

df.replace('', np.nan, inplace=True)
df['Code'].fillna(df['State'].map(state_code_dict), inplace=True)
df['State'].fillna(df['Code'].map(code_state_dict), inplace=True)

print(df)

       State Code
0    Alabama   AL
1     Alaska   AK
2    Arizona   AZ
3   Arkansas   AR
4       Iowa   IA
5     Hawaii   HI
6      Idaho   ID
7    Alabama   AL
8       Iowa   IA
9     Alaska   AK
10   Arizona   AZ

Answer 3

To fill in the codes

df['Code'] = df.apply(lambda x: x['Code'] if x['Code']!='' else state_code_dict[x['State']],axis=1)

To fill in the states

state_code_dict2 = {v: k for k, v in state_code_dict.items()}
df['State'] = df.apply(lambda x: x['State'] if x['State']!='' else state_code_dict2[x['Code']],axis=1)

Answer 4

Similar question to Filling a series based on key value pairs

Using your data:

(df.replace('', np.nan)
  .sort_values(by=['State', 'Code'], ascending=False)
  .groupby('State').ffill().bfill()
  .groupby('Code').ffill().bfill())

Output:

    Code    State
4   IA  Iowa
6   ID  Idaho
5   HI  Hawaii
3   AR  Arkansas
2   AZ  Arizona
1   AK  Alaska
9   AK  Alaska
0   AL  Alabama
7   AL  Alabama
8   IA  Iowa
10  AZ  Arizona