Pandas is there a Pythonic way to add a new hierarchical column to flat column Dataframe

Question

In Pandas, manipulating hierarchical column indexes feels harder than it needs to be, so I assume I must be missing something.

For example.

pd.DataFrame(dict(A=1, B=2, C=3), index=pd.Index(['key1', 'key2'], name='pk'))

       A  B  C
 pk           
 key1  1  2  3
 key2  1  2  3

I want to add a new column, but under a new hierarchy.

df['levelone', 'levelone - d'] = 4

Adds a tuple key ('levelone', 'levelone-d) to the flat column index, but does not append a new leveled hierarchy. I cannot access this column using df['levelone'] .

Is there a better way to do this than to use something hacky like

multiindex = pd.MultiIndex.from_tuples(
[(col_name, '') if isinstance(col_name, str) else col_name for col_name in df.columns]
)
df.columns = multiindex

This produces the output I want

       A  B  C     levelone
               levelone - d
 pk                        
 key1  1  2  3            4
 key2  1  2  3            4

But it feels bad. Why is raising and lowering hierarchies such a pain. What am I missing?

Answer 1

Let us try

df.columns=pd.MultiIndex.from_product([df.columns,['']])
df.loc[:,('levelone', 'levelone - d')]=4
df
      A  B  C     levelone
              levelone - d
pk                        
key1  1  2  3            4
key2  1  2  3            4