pandas get_dummies() for multiple columns with a pre-defined list

Question

I'm struggling with creating columns of dummies for my dataframe.

This is my original dataframe:

df = pd.DataFrame({'id': ['01', '02', '03'],
                    'Q1': ['a', 'b', 'a'],
                    'Q2': ['c', 'b', 'a']})
print(df)

   id Q1 Q2
0  01  a  c
1  02  b  b
2  03  a  a

I have a pre-defined list of answers for both Q1 and Q2:

ls = list("abc")
print(ls)
['a', 'b', 'c']

My expected structure of dataframe:

   id Q1_a Q1_b Q1_c Q2_a Q2_b Q2_c
0  01    1    0    0    0    0    1
1  02    0    1    0    0    1    0
2  03    1    0    0    1    0    0

Please help! Thanks!

Answer 1

Based on the post here , here is one answer:

df2 = pd.get_dummies(df[['Q1', 'Q2']].astype(pd.CategoricalDtype(categories=ls)))
df2.insert(0, 'id', df['id'])

Output:

df2
    id  Q1_a    Q1_b    Q1_c    Q2_a    Q2_b    Q2_c
0   01     1       0       0       0       0    1
1   02     0       1       0       0       1    0
2   03     1       0       0       1       0    0

Answer 2

Try:

df = pd.DataFrame(
    {"id": ["01", "02", "03"], "Q1": ["a", "b", "a"], "Q2": ["c", "b", "a"]}
)

ls = list("abc")

idx = pd.MultiIndex.from_product([df.loc[:, "Q1":], ls])
x = pd.concat(
    {c: pd.get_dummies(df[c]) for c in df.loc[:, "Q1":]}, axis=1
).reindex(columns=idx, fill_value=0)
x.columns = x.columns.map("_".join)
print(pd.concat([df["id"], x], axis=1))

Prints:

   id  Q1_a  Q1_b  Q1_c  Q2_a  Q2_b  Q2_c
0  01     1     0     0     0     0     1
1  02     0     1     0     0     1     0
2  03     1     0     0     1     0     0