Python return second highest value column name in row

Question

data

data = [['john', 0.20, 0.0, 0.4, 0.40],['katty', 0.0, 1.0, 0.0, 0.0],['kent', 0.0, 0.51, 0.49, 0.0]]
df = pd.DataFrame(data, columns=['name','fruit', 'vegetable', 'softdrinks', 'icecream'])
df = df.set_index('name')
df.head()

desired outcome

data = [['john', 0.20, 0.0, 0.4, 0.40,'softdrinks','icecream'],['katty', 0.0, 1.0, 0.0, 0.0,'vegetable','NaN'],['kent', 0.0, 0.51, 0.49, 0.0,'vegetable','softdrinks']]
df = pd.DataFrame(data, columns=['name','fruit', 'vegetable', 'softdrinks', 'icecream', 'max_no1', 'max_no2'])
df = df.set_index('name')
df.head()

tried idxmax which only return highest value column name, i need to locate the second highest row value column name, how to achieve this?

thanks a lot

Answer 1

First set 0 to missing values by DataFrame.mask , then reshape by DataFrame.stack and for top2 use SeriesGroupBy.nlargest , last DataFrame.join reshaped data by DataFrame.pivot :

df1 = df.mask(df == 0).stack().groupby(level=0, group_keys=False).nlargest(2).reset_index()
df1 = df1.assign(a = df1.groupby('name').cumcount().add(1))

df = df.join(df1.pivot('name','a','level_1').add_prefix('max_no'))
print (df)
       fruit  vegetable  softdrinks  icecream     max_no1     max_no2
name                                                                 
john     0.2       0.00        0.40       0.4  softdrinks    icecream
katty    0.0       1.00        0.00       0.0   vegetable         NaN
kent     0.0       0.51        0.49       0.0   vegetable  softdrinks

Or solution from comment with DataFrame.idxmax and again set missing values by compare with broadcasting in numpy :

df1 = df.mask(df == 0)
df['max_no1'] = df1.idxmax(axis=1)
m = df1.columns.to_numpy() == df['max_no1'].to_numpy()[:, None]
#pandas below 0.24
#m = df1.columns.values == df['max_no1'].values[:, None]
df1 = df1.mask(m)
df['max_no2'] = df1.idxmax(axis=1)
print (df)
       fruit  vegetable  softdrinks  icecream     max_no1     max_no2
name                                                                 
john     0.2       0.00        0.40       0.4  softdrinks    icecream
katty    0.0       1.00        0.00       0.0   vegetable         NaN
kent     0.0       0.51        0.49       0.0   vegetable  softdrinks