连接 dataframe 中除 NaN 之外的所有列

Question

Another simple one. I have a DataFrame (1056 x 39) that contains reference variables from a pivot table. I now need to generate a column of concatenated values of all columns, which exclude NaNs. The trouble is that I have quite a few NaNs which are interfering with the output.

Based on another post that I have found Concatenating all columns in pandas dataframe , I can use this approach.

df['Merge'] = df.astype(str).agg(' or '.join,axis=1)

The trouble is that NaNs remain. How can I modify this line to exclude NaN values (skip them essentially) such that the output will only contain concatenated values.

The intended output should appear as (first row):

df['Merge'][0] = 'Var1 or Var2 or Var 20 or Var28' (all NaN values were excluded)

Thanks:)

Answer 1

You can stack to remove the NaN then cast to string and groupby + str.join

import pandas as pd
df = pd.DataFrame([[1.0, np.NaN, 2, 3, 'foo'], [np.NaN, None, 5, 'bar', 'bazz']])

df['merged'] = df.stack().astype(str).groupby(level=0).agg(' or '.join)
#     0   1  2    3     4                merged
#0  1.0 NaN  2    3   foo  1.0 or 2 or 3 or foo
#1  NaN NaN  5  bar  bazz      5 or bar or bazz

---

Or you can apply along the rows, dropping nulls, casting to string then joining all the non-nulls.

df = pd.DataFrame([[1.0, np.NaN, 2, 3, 'foo'], [np.NaN, None, 5, 'bar', 'bazz']])

df['merged'] = df.apply(lambda row: ' or '.join(row.dropna().astype(str)), axis=1)
#     0   1  2    3     4                merged
#0  1.0 NaN  2    3   foo  1.0 or 2 or 3 or foo
#1  NaN NaN  5  bar  bazz      5 or bar or bazz