Pandas PivotTable

Question

I have a Pandas dataframe with the following columns:

SecId   Date           Sector     Country
184149  2019-12-31     Utility    USA
184150  2019-12-31     Banking    USA
187194  2019-12-31     Aerospace  FRA
...............
128502  2020-02-12     CommSvcs   UK
...............

SecId & Date columns are the indices. What I want is the following..

SecId   Date          Aerospace Banking CommSvcs ........  Utility AFG CAN .. FRA .... UK   USA ...
184149  2019-12-31        0        0       0                  1     0   0      0        0    1
184150  2019-12-31        0        1       0                  0     0   0      0        0    1
187194  2019-12-31        1        0       0                  0     0   0      1        0    0
................
128502  2020-02-12        0        0       1                  0     0   0      0        1    0
................

What is the efficient way to pivot this? The original data is denormalized for each day and can have millions of rows.

Answer 1

You can use get_dummies . You can cast as a categorical dtype beforehand to define what columns will be created.

code:

SECTORS = df.Sector.unique()
df["Sector"] = df.Sector.astype(pd.Categorical(SECTORS))
COUNTRIES = df.Country.unique()
df["Country"] = df.Country.astype(pd.Categorical(COUNTRIES))

df2 = pd.get_dummies(data=df, columns=["Sector", "Country"], prefix="", pefix_sep="")

output:

    SecId        Date  Aerospace  Banking  Utility  FRA  USA
0  184149  2019-12-31          0        0        1    0    1
1  184150  2019-12-31          0        1        0    0    1
2  187194  2019-12-31          1        0        0    1    0

Answer 2

Try as @BEN_YO suggests:

pd.get_dummies(df,columns=['Sector', 'Country'], prefix='', prefix_sep='')

Output:

    SecId        Date  Aerospace  Banking  Utility  FRA  USA
0  184149  2019-12-31          0        0        1    0    1
1  184150  2019-12-31          0        1        0    0    1
2  187194  2019-12-31          1        0        0    1    0