How to rename the duplicated MultiIndex column names?

Question

I have a dataframe with two levels of columns index.

Reproducible Dataset.

df = pd.DataFrame(
   [ ['Gaz','Gaz','Gaz','Gaz'],
    ['X','X','X','X'],
    ['Y','Y','Y','Y'],
    ['Z','Z','Z','Z']],
columns=pd.MultiIndex.from_arrays([['A','A','C','D'],
                          ['Name','Name','Company','Company']])

I want to rename the duplicated MultiIndex columns, only when level-0 and level-1 combined is duplicated. Then add a suffix number to the end. Like the one below.

Below is a solution I found, but it only works for single level column index.

class renamer():
def __init__(self):
    self.d = dict()

def __call__(self, x):
    if x not in self.d:
        self.d[x] = 0
        return x
    else:
        self.d[x] += 1
        return "%s_%d" % (x, self.d[x])
df = df.rename(columns=renamer())

I think the above method can be modified to support the multi level situation, but I am too new to pandas/python.

Thanks in advance.

@Datanovice This is to clarify to you about the output what I need. I have the snippet below.

import pandas as pd
import numpy as np

df = pd.DataFrame(
   [ ['Gaz','Gaz','Gaz','Gaz'],
    ['X','X','X','X'],
    ['Y','Y','Y','Y'],
    ['Z','Z','Z','Z']],
columns=pd.MultiIndex.from_arrays([
                        ['A','A','C','A'], 
                        ['A','A','C','A'],
                        ['Company','Company','Company','Name']]))

s = pd.DataFrame(df.columns.tolist())
cond = s.groupby(0).cumcount()

s = [np.where(cond.gt(0),s[i] + '_' + cond.astype(str),s[i]) for i in 
range(df.columns.nlevels)]
s = pd.DataFrame(s)
#print(s)


df.columns = pd.MultiIndex.from_arrays(s.values.tolist())

print(df)

The current result is-

What I need is the last piece of column index should not be counted as duplicated, as as "AA-Name" is not same with the first two.

Thank you again.

Answer 1

Might be a better way to do this, but you could return a dataframe from your columns and apply a conditional operation on them and re-assign them.

df = pd.DataFrame(
   [ ['Gaz','Gaz','Gaz','Gaz'],
    ['X','X','X','X'],
    ['Y','Y','Y','Y'],
    ['Z','Z','Z','Z']],
columns=pd.MultiIndex.from_arrays([['A','A','C','A'],
                          ['Name','Name','Company','Company']])


s = pd.DataFrame(df.columns.tolist())

cond = s.groupby([0,1]).cumcount()

s[0] = np.where(cond.gt(0),s[0] + '_' + cond.astype(str),s[0])
s[1] = np.where(cond.gt(0),s[1] + '_' + cond.astype(str),s[1])

df.columns = pd.MultiIndex.from_frame(s)

print(df)

0    A    A_1       C       D
1 Name Name_1 Company Company
0  Gaz    Gaz     Gaz     Gaz
1    X      X       X       X
2    Y      Y       Y       Y
3    Z      Z       Z       Z

Answer 2

Try this -

arrays = [['A', 'A', 'A', 'A', 'B', 'B', 'B', 'B'],['A', 'A', 'A', 'B', 'C', 'C', 'D', 'D']]
tuples = list(zip(*arrays))

index = pd.MultiIndex.from_tuples(tuples)
df = pd.DataFrame(np.random.randn(3, 8), columns=index)

    A               B
    A   A   A   B   C   C   D   D
0   0   0   1   3   1   2   1   4
1   0   1   1   1   1   3   0   1
2   1   1   4   2   3   2   1   4

suffix = pd.DataFrame(df.columns)
suffix['count'] = suffix.groupby(0).cumcount()
suffix['new'] = [((i[0]+'_'+str(j)),(i[1]+'_'+str(j))) for i,j in zip(suffix[0],suffix['count'])]
new_index = pd.MultiIndex.from_tuples(list(suffix['new']))
df.columns = new_index