Transpose multiple columns in pairs of two - pandas python

Question

I would like to transpose multiple columns in pairs of two

I have the following columns:

user_id', 'fullname', 'email', 'handle', 'audience_ethnicities_code0', 'audience_ethnicities_weight0', 'audience_ethnicities_code1', 'audience_ethnicities_weight1', 'audience_ethnicities_code2', 'audience_ethnicities_weight2', 'audience_ethnicities_code3', 'audience_ethnicities_weight3'

where code and weigh are related, for example:

user_id = ABCD

'audience_ethnicities_code0' = asian;
'audience_ethnicities_weight0' = 0.4 

'audience_ethnicities_code1' = african; 
'audience_ethnicities_weight1' = 0.2

'audience_ethnicities_code2' = white;
'audience_ethnicities_weight2' = 0.2 

'audience_ethnicities_code3' = hispanic; 
'audience_ethnicities_weight3' = 0.2

tot weight = 1, and the audience of user ABCD is 40% Asian, 20% African etc. What I want is to have ethnicity ( audience_ethnicities_code_n ) in the column and in the row their weight ( audience_ethnicities_weight_n ) for each user

I tried this query but it gave me a messy result:

df1 = df.pivot_table(index=['user_id', 'fullname', 'email', 'handle'], 
                    columns=['audience_ethnicities_code0', 'audience_ethnicities_code1', 'audience_ethnicities_code2', 'audience_ethnicities_code3'], 
                    values=['audience_ethnicities_weight0', 'audience_ethnicities_weight1', 'audience_ethnicities_weight2', 'audience_ethnicities_weigh3'], aggfunc=lambda x: ' '.join(str(v) for v in x))

df1

any ideas?

Answer 1

I would iteratively do the pivot for each column and then merge the dataframes by their index.

Here an example:

from functools import reduce

index = ['user_id', 'fullname', 'email', 'handle']

dfList = []
for i in range(3):
  dfList.append(df.pivot_table(index=index, 
                               columns='audience_ethnicities_code{}'.format(i), 
                               values='audience_ethnicities_weight{}'.format(i))
                  .rename_axis(None, axis=1)
                  .reset_index())

reduce(lambda x, y: pd.merge(x, y, on=index), dfList)