聚合pandas数据帧中的单元格/列

Question

I have a dataframe that is like this

Index Z1       Z2       Z3       Z4  
 0    A(Z1W1)  A(Z2W1)  A(Z3W1) B(Z4W2)   
 1    A(Z1W3)  B(Z2W1)  A(Z3W2) B(Z4W3)   
 2    B(Z1W1)           A(Z3W4) B(Z4W4)
 3    B(Z1W2)

I want to convert it to

Index   Z1              Z2        Z3                    Z4
 0      A(Z1W1,Z1W3)    A(Z2W1)   A(Z3W1,Z3W2,Z3W4)     B(Z4W2,Z4W3,Z4W4)    
 1      B(Z1W1,Z1W2)    B(Z2W1)     

Basically I want to aggregate the values of different cell to one cell as shown above

Edit 1

Actual column names are either two words or 3 words names and not AB For example Nut Butter instead of A

Answer 1

Things are getting interested : -)

s=df.stack().replace({'[(|)]':' '},regex=True).str.strip().str.split(' ',expand=True)
v=('('+s.groupby([s.index.get_level_values(1),s[0]])[1].apply(','.join)+')').unstack().apply(lambda x : x.name+x.astype(str)).T
v[~v.apply(lambda x : x.str.contains('None'))].apply(lambda x : sorted(x,key=pd.isnull)).reset_index(drop=True)
Out[1865]: 
             Z1       Z2                 Z3                 Z4
0  A(Z1W1,Z1W3)  A(Z2W1)  A(Z3W1,Z3W2,Z3W4)  B(Z4W2,Z4W3,Z4W4)
1  B(Z1W1,Z1W2)  B(Z2W1)                NaN                NaN

Update Change

#s=df.stack().replace({'[(|)]':' '},regex=True).str.strip().str.split(' ',expand=True)

to

s=df.stack().str.split('(',expand=True)
s[1]=s[1].replace({'[(|)]':' '},regex=True).str.strip()

Answer 2

Geneal idea:

1. split string values
2. regroup and join stings
3. apply to all columns

---

Update 1

# I had to add parameter as_index=False to groupby(0) 
# to get exactly same output as asked

Lets try one column

def str_regroup(s):
return s.str.extract(r"(\w)\((.+)\)",expand=True).groupby(0,as_index=False).apply(
    lambda x: '{}({})'.format(x.name,', '.join(x[1])))
str_regroup(df1.Z1)

output

A   A(Z1W1, Z1W3)
B   B(Z1W1, Z1W2)

then apply to all columns

df.apply(str_regroup)

output

    Z1  Z2  Z3  Z4
0   A(Z1W1, Z1W3)   A(Z2W1) A(Z3W1, Z3W2, Z3W4) B(Z4W2, Z4W3, Z4W4)
1   B(Z1W1, Z1W2)   B(Z2W1)     

---

Update 2
Performance on 100 000 sample rows

• 928 ms for this apply version ;b
• 1.55 s for stack() by @Wen

Answer 3

You could use the following approach:

• Melt df to get:

   In [194]: melted = pd.melt(df, var_name='col'); melted Out[194]: col value 0 Z1 A(Z1W1) 1 Z1 A(Z1W3) 2 Z1 B(Z1W1) 3 Z1 B(Z1W2) 4 Z2 A(Z2W1) 5 Z2 B(Z2W1) 6 Z2 7 Z2 8 Z3 A(Z3W1) 9 Z3 A(Z3W2) 10 Z3 A(Z3W4) 11 Z3 12 Z4 B(Z4W2) 13 Z4 B(Z4W3) 14 Z4 B(Z4W4) 15 Z4 

• Use regex to extract row and value columns:

   In [195]: melted[['row','value']] = melted['value'].str.extract(r'(.*)\\((.*)\\)', expand=True); melted Out[195]: col value row 0 Z1 Z1W1 A 1 Z1 Z1W3 A 2 Z1 Z1W1 B 3 Z1 Z1W2 B 4 Z2 Z2W1 A 5 Z2 Z2W1 B 6 Z2 NaN NaN 7 Z2 NaN NaN 8 Z3 Z3W1 A 9 Z3 Z3W2 A 10 Z3 Z3W4 A 11 Z3 NaN NaN 12 Z4 Z4W2 B 13 Z4 Z4W3 B 14 Z4 Z4W4 B 15 Z4 NaN NaN 

• Group by col and row and join the value s together:

   In [185]: result = melted.groupby(['col', 'row'])['value'].agg(','.join) In [186]: result Out[186]: col row Z1 A Z1W1,Z1W3 B Z1W1,Z1W2 Z2 A Z2W1 B Z2W1 Z3 A Z3W1,Z3W2,Z3W4 Z4 B Z4W2,Z4W3,Z4W4 Name: value, dtype: object 

• Add the row values to the value values:

   In [188]: result['value'] = result['row'] + '(' + result['value'] + ')' In [189]: result Out[189]: row value col Z1 AA(Z1W1,Z1W3) Z1 BB(Z1W1,Z1W2) Z2 AA(Z2W1) Z2 BB(Z2W1) Z3 AA(Z3W1,Z3W2,Z3W4) Z4 BB(Z4W2,Z4W3,Z4W4) 

• Overwrite the row column values with groupby/cumcount values to setup the upcoming pivot:

   In [191]: result['row'] = result.groupby(level='col').cumcount() In [192]: result Out[192]: row value col Z1 0 A(Z1W1,Z1W3) Z1 1 B(Z1W1,Z1W2) Z2 0 A(Z2W1) Z2 1 B(Z2W1) Z3 0 A(Z3W1,Z3W2,Z3W4) Z4 0 B(Z4W2,Z4W3,Z4W4) 

• Pivoting produces the desired result:

   result = result.pivot(index='row', columns='col', values='value') 

---

import pandas as pd
df = pd.DataFrame({
 'Z1': ['A(Z1W1)', 'A(Z1W3)', 'B(Z1W1)', 'B(Z1W2)'],
 'Z2': ['A(Z2W1)', 'B(Z2W1)', '', ''],
 'Z3': ['A(Z3W1)', 'A(Z3W2)', 'A(Z3W4)', ''],
 'Z4': ['B(Z4W2)', 'B(Z4W3)', 'B(Z4W4)', '']}, index=[0, 1, 2, 3],)

melted = pd.melt(df, var_name='col').dropna()
melted[['row','value']] = melted['value'].str.extract(r'(.*)\((.*)\)', expand=True)
result = melted.groupby(['col', 'row'])['value'].agg(','.join)
result = result.reset_index('row')
result['value'] = result['row'] + '(' + result['value'] + ')'
result['row'] = result.groupby(level='col').cumcount()
result = result.reset_index()
result = result.pivot(index='row', columns='col', values='value')
print(result)

yields

col            Z1       Z2                 Z3                 Z4
row                                                             
0    A(Z1W1,Z1W3)  A(Z2W1)  A(Z3W1,Z3W2,Z3W4)  B(Z4W2,Z4W3,Z4W4)
1    B(Z1W1,Z1W2)  B(Z2W1)                NaN                NaN