Making multiple columns in pandas based on index

Question

I'm trying to set a pandas dataframe column into an index. Since it is a categorical column, I would like to collapse all column in same category. For example:

df

  col1   Set_Index    col2
0    1        A         'a'
1    2        B         'b'
2    3        A         'c'
3    4        B         'd'

The new df must look like:

Edit: I forgot to put the default index in my first draft of the output needed in final output. It must be finally look like:

new_df

             col1      col2
Set_Index 
   A      0    1       'a'
          2    3       'c'
   B      1    2       'b'
          3    4       'd'

I tried to use the groupby() function but it didn't help much.

Answer 1

You can set the 'Set_Index' column as index, drop it and sort the rows based on the new indices.

df_new = df.set_index(df.Set_Index).drop(columns=['Set_Index']).sort_index()
df_new

And you will have the following df_new:

As far as I know, every row should have an index and I would not know how to create the df_new you indicated (without index for row 2 and 4).

You can read more about indexing in pandas here: https://brohrer.github.io/dataframe_indexing.html