如果列名 == 年份且值为 NaN 熊猫，则将数据框中的值向左移动

Question

I have a dataframe that looks like this:

                              0    1   2018       3   2017       5
0                    Population    3    NaN  418980    NaN  501433
1                       British    4  31514     NaN  96797     NaN
2                        French  NaN   3089     NaN    201     NaN
3                           NaN  NaN  34603     NaN  96998     NaN

I want to end up with a dataframe that looks like this:

                              0    1   2018       3   2017       5
0                    Population    3  418980    NaN  501433    NaN
1                       British    4  31514     NaN  96797     NaN
2                        French  NaN   3089     NaN    201     NaN
3                           NaN  NaN  34603     NaN  96998     NaN

Where the logic is: If a year column has a NaN value, look to the right for a numerical value and replace the NaN value.

I believe I need to find the index of any year column, look for df['2018'].isnull() , if it is null, add one to the index then search for the corresponding value but am unsure if this is the best method.

Answer 1

pandas has a built in function for using another column to replace the NA values in the original :

df[2018] = df[2018].combine_first(df[3])

If you have many columns like that, think how to loop over the columns to use the column name and it's right-sided one's name. (or I can help you with that)

Answer 2

Idea is replace next values of years to years with forward filling misisng values and then use DataFrame.groupby with axis=1 for grouping per columns and get first non missing values if exist by GroupBy.first :

s = df.columns.astype(str).to_series()
a = s.where(s.str.contains('\d{4}')).ffill().fillna(s)
print (a)
0          0
1          1
2018    2018
3       2018
2017    2017
5       2017
dtype: object

df1 = df.groupby(pd.Index(a), axis=1).first()
print (df1)
         0     1         2017      2018
0  Population   3.0  501433.0  418980.0
1     British   4.0   96797.0   31514.0
2      French   NaN     201.0    3089.0
3         NaN   NaN   96998.0   34603.0

Answer 3

By using what @Aryerez answered with I came up with this:

columns_list = list(df.columns) 
year_column_indexes = [i for i, item in enumerate(columns_list) if re.search('201[0-9]', item)]
for _index in year_column_indexes:
    df.iloc[:, _index] = df.iloc[:, _index].combine_first(df.iloc[:, _index+1])
    df = df.drop(df.columns[_index+1], axis=1)

But it needs a bit of editing.