Combining multiple pivot tables horizontally and vertically in pandas

Question

I have a dataset that looks like this:

raw data

and I need to get this output: desired output

Is using pivot tables (in python pandas) first for each Q, then appending over each column, and then merging the resulting dataframes for reach Q the best way to do it? Any suggestion on how to do that to get the right format?

Answer 1

Try this:

df1 = pd.melt(df, id_vars=["gender",'region'], value_vars=['q1','q2'], value_name='ques')
df2 = pd.melt(df1, id_vars='ques', value_vars=["gender",'region'], value_name='col')
df3 = pd.pivot_table(df2,index='ques',columns=['col'], aggfunc='count').fillna(0)
print(df3)

    col                             EE   LA   WE female male
ques                                                         
1'Agree + strongly agree            5.0  1.0  4.0    6.0  4.0
2'neither agree nor disagree        0.0  0.0  1.0    1.0  0.0
3'disagree + strongly disagree      0.0  3.0  0.0    3.0  0.0
no                                  5.0  3.0  4.0    8.0  4.0
yes                                 0.0  1.0  1.0    2.0  0.0