Pandas Most Recent “N” Non NA values

Question

I have a dataframe:

2019-03-13 11:30:00+08:00       NaN       NaN  0.001143
2019-03-13 15:00:00+08:00       NaN       NaN       NaN
2019-03-14 01:00:00+08:00  0.003653       NaN       NaN
2019-03-14 10:15:00+08:00       NaN -0.002743       NaN
2019-03-14 11:30:00+08:00       NaN       NaN  0.000229
2019-03-14 15:00:00+08:00       NaN       NaN       NaN
2019-03-15 01:00:00+08:00 -0.000229       NaN       NaN
2019-03-15 10:15:00+08:00       NaN  0.003211       NaN
2019-03-15 11:30:00+08:00       NaN       NaN -0.006192
2019-03-15 15:00:00+08:00       NaN       NaN       NaN

Is there a way to get the most recent N=2 values for each column without looping? ie, skipping all the NaN . There exist a last_valid_index() , but that only gets the last value. It would be nice to get a reindex dataframe abset of the datetime so they are aligned. Is this possible?

Expected Output:

   1  0.003653  -0.002743     0.000229
   2  -0.000229  0.003211    -0.006192

Answer 1

IIUC

df.apply(lambda x : sorted(x,key=pd.notnull)).iloc[-2:]
                   1         2         3
2019-03-15  0.003653 -0.002743  0.000229
2019-03-15 -0.000229  0.003211 -0.006192