Python pandas：替换时间序列中的缺失值

Question

I am currently working on a routine to process meteodata from different weatherstations. Unfortunetly there is missing data from time to time. I wrote a routine to merge data from all stations into a dataframe and implemented a "NaN_Flag" column which shows missing data of a certain timepoint.

Now the plan is to fill those data gaps with data of a nearby station. For temperature and humidity interpolation would be possible but depending on size of the gap not really ideal. For rain events interpolation wouldnt make any sense.

The first column is the index column containing date,time and location. Now I am looking for a solution to fill in meassured data of another location (same time) if the "NaN_Flag" shows "1".

So in the following simplified example I would like that the dataset of 01-01-01 00:20:00 of Location1 is automaticly replaced with data of the same datetime ofLocation2. So every Location has a "backup"-Location and everytime the "NaN_Flag" shows "1" data is automaticly replaced with the the appropriate backup data. Anyone got any idea how to accomplish that?

DATETIME_UTC_LOCATION           DATETIME_UTC              LOCATION    TEMP   PLUV   HUM   NaN_FLAG 
2020-01-01 00:00:00 Location1   2020-01-01 00:00:00       Location1   5.25   0.0    87.3  0
2020-01-01 00:10:00 Location1   2020-01-01 00:10:00       Location1   6.12   0.1    85.0  0
2020-01-01 00:20:00 Location1   2020-01-01 00:20:00       Location1                       1
2020-01-01 00:00:00 Location2   2020-01-01 00:00:00       Location2   5.12   0.0    88.9  0
2020-01-01 00:10:00 Location2   2020-01-01 00:10:00       Location2   6.25   0.1    84.3  0
2020-01-01 00:20:00 Location2   2020-01-01 00:20:00       Location2   6.75   0.2    82.5  0

Answer 1

If the dataframe you have has an equivalent format to this one:

import pandas as pd
import numpy as np


df = pd.DataFrame(data={'month': ["Jan","Feb","Mar","Jan","Feb","Mar"],
                        'station': ["station_1","station_1","station_1","station_2","station_2","station_2"],
                        'values': [3.2, np.nan, 4.1, 3.6, 5.8, 4.2]}).set_index('month')

output:

              station   values
    month       
    Jan       station_1    3.2
    Feb       station_1    NaN
    Mar       station_1    4.1
    Jan       station_2    3.6
    Feb       station_2    5.8
    Mar       station_2    4.2

You can use:

df.loc[df['station'] == "station_1"] = df.loc[df['station'] == "station_1"].fillna(df.loc[df['station'] == "station_2"])

to substitute NaN values of the station 1 for the station 2 equivalent values. By "equivalent" I mean matching in the "month" index.

Output:

              station   values
    month       
    Jan       station_1    3.2
    Feb       station_1    5.8
    Mar       station_1    4.1
    Jan       station_2    3.6
    Feb       station_2    5.8
    Mar       station_2    4.2