Error in merging pandas data frame columns

Question

I'm trying to merge three columns from the same data frame into one.

Here my data frame selected_vals

   label_1                         label_2                   label_3   
0  NaN                              NaN                      NaN
1  ('__label__Religione_e_Magia',)  NaN                      NaN
2  NaN                            ('__label__Storia',)       NaN
3  NaN                            ('__label__Storia',)       NaN
4 ('__label__Religione_e_Magia',)  NaN                       NaN

The dataframe has only one value per row so, in the col where the value it's not specified I'm having NaN Following the solution proposed here I used this code:

selected_vals['selected_vals'] =  selected_vals.loc[:,selected_vals.columns.tolist()[1:]].apply(lambda x: x.dropna().tolist(), 1)

However, by doing so, only the values from the col label_2 are in the col selected_vals

Here the ouput

 label_1                         label_2                   label_3  selected_vals   
0  NaN                              NaN                      NaN      []
1  ('__label__Religione_e_Magia',)  NaN                      NaN      []
2  NaN                            ('__label__Storia',)       NaN      ('__label__Storia',)
3  NaN                            ('__label__Storia',)       NaN      ('__label__Storia',)
4 ('__label__Religione_e_Magia',)  NaN

As desired output I would like to have all the values stored in the same col ie

   selected_vals                              
0  NaN                              
1  ('__label__Religione_e_Magia',)  
2  ('__label__Storia',)                                   
3  ('__label__Storia',)                            
4 ('__label__Religione_e_Magia',)  

Suggestions about how to deal with this problem?

Thanks

Answer 1

Use DataFrame.iloc for select all columns without first, then forward fiiling missing values and last select last column:

#replace NaN strings to np.nan if necessary
selected_vals = selected_vals.replace('NaN', np.nan)

selected_vals['selected_vals'] =  selected_vals.iloc[:,1:].ffill(axis=1).iloc[:, -1]

Answer 2

You can apply function to each row and keep only desired value (where column is not NaN)

selected_vals['selected_vals'] = selected_vals.apply(lambda row: row[row[pd.notnull(row)].index.item()], axis=1)

Answer 3

Thanks for your suggestions.

I think the problem was related to the type of the dataframe.

I solved the issue as follows:

selected_vals = selected_vals.replace(np.nan, '', regex=True)
selected_vals = selected_vals.applymap(str)
df['suggested_label'] = selected_vals["label_1"].astype(str) + selected_vals["label_2"]+ selected_vals["label_3"]

print(df)

Don't know if it's correct or not but at least it works for me.