如何在熊猫中将n列合并为1行

Question

I have a DataFrame containing ~40 columns and over 150K rows that I want to consolidate into just one column. The DataFrame has NaN values all over the place.

Here's an example of what my df looks like:

d = {'A' : pd.Series([np.nan, 5., 3.], index=[0,1,2]),
 'B' : pd.Series([np.nan, 2., np.nan], index=[0,1,2]),
 'C' : pd.Series([1.,np.nan, 4.], index=[0,1,2])}
df = pd.DataFrame(d)

     A    B    C
0  NaN  NaN  1.0
1  5.0  2.0  NaN
2  3.0  NaN  4.0

I want my resulting df to contain all the values from all columns, but only one column. The rows can have multiple values in multiple columns, so I need a way to keep them all like so:

e = {'ABC' : pd.Series([1.,5.,2.,3.,4.], index=[0,1,2,3,4])}
df1 = pd.DataFrame(e)
   ABC
0  1.0
1  5.0
2  2.0
3  3.0
4  4.0

The column names are all different so I haven't been able to join, merge or concatenate them.

Thanks in advance!

Answer 1

Maybe stack seems to be a good option which automatically drops all NaNs by default leaving with just finite entries:

pd.DataFrame({'ABC': df.stack().values})