python pandas groupby多列到一行

Question

I am wanting to group a dataframe on a key, in this case clientid. And to have all columns of that grouped concatenated into one long row for each key.

eg

clientid, name, age, company
1,        tom,  31,  awesome
1,        jen,  26,  argos
2,        bob,  18,  hmv
3,        ted,  12,  mcdonalds
4,        sarah,30,  MnS
4,        mike, 52,  Mns
4,        luke, 75,  argos

wanted result

clientid, name, age, company,  name, age, company, name, age, company
1,        tom,  31,  awesome,  jen,  26,  argos,
2,        bob,  18,  hmv,
3,        ted,  12,  mcdonadls,
4,        sarah,30,  MnS,      mike, 52,  MnS,     luke, 75,  argos,

A similar question and solution was given

df_info = df1.groupby('clientid')['info'].unique().apply(pd.Series).reset_index()
info_len = len([col for col in df_info if str(col).isdigit()])
df_info.columns = ['clientid'] + ['info'] * info_len
df_info

But I can't find how to apply this to multiple columns

Answer 1

This comes with a health warning, as you are losing much of the power of pandas by leaving the structure allowing you to groupby, the great performance, the powerful and clean syntax (so in some sense it's a feature you can't do this easily!)... and it's just not very pandorable.

So I strongly suggest not doing this, as there is almost certainly a better way to do whatever it is you're doing...

---

I think you need to groupby the clientid and then extract these strings...

In [11]: df1 = df.set_index('clientid')

In [12]: df1
Out[12]:
           name  age    company
clientid
1           tom   31    awesome
1           jen   26      argos
2           bob   18        hmv
3           ted   12  mcdonalds
4         sarah   30        MnS
4          mike   52        Mns
4          luke   75      argos

In [13]: g = df1.groupby(df1.index)

I would probably look into using to_csv over each group:

In [14]: g.apply(lambda x: x.to_csv(header=False, index=False, line_terminator=','))
Out[14]:
clientid
1                      tom,31,awesome,jen,26,argos,
2                                       bob,18,hmv,
3                                 ted,12,mcdonalds,
4           sarah,30,MnS,mike,52,Mns,luke,75,argos,
dtype: object

An alternative is to apply:

In [15]: g.apply(lambda x: pd.concat([row for _, row in x.iterrows()]).values)
Out[15]:
clientid
1                         [tom, 31, awesome, jen, 26, argos]
2                                             [bob, 18, hmv]
3                                       [ted, 12, mcdonalds]
4           [sarah, 30, MnS, mike, 52, Mns, luke, 75, argos]
dtype: object

You have to hack this a little to get the correct header:

In [16]: list(df1.columns) * g.apply(len).max()
Out[16]: ['name', 'age', 'company', 'name', 'age', 'company', 'name', 'age', 'company']

So, you can do something like the following:

In [21]: s = g.apply(lambda x: pd.concat([row for _, row in x.iterrows()]).values).apply(lambda row: ','.join([str(x) for x in row]))

In [22]: s.name = ','.join(list(df1.columns) * g.apply(len).max())

In [23]: s.to_frame().to_csv(quotechar=" ")  # Note: this is a hack since quoting=0 seems to be ignored
Out[23]: 'clientid, name,age,company,name,age,company,name,age,company \n1, tom,31,awesome,jen,26,argos \n2, bob,18,hmv \n3, ted,12,mcdonalds \n4, sarah,30,MnS,mike,52,Mns,luke,75,argos \n'

Answer 2

I have tried a couple of approaches and come up with a altered version of Andy's that I found works well.

grouped = df1.groupby('clientid')
flattenedSeries = grouped.apply(lambda x: x.to_csv(header=False, index=False, line_terminator=','))
flattenedSeries = pd.DataFrame(flattenedSeries, columns=['data'])
ready = flattenedSeries['data'].apply(lambda x: pd.Series(x.split(',')))

Create new column headers

newcolumns = list(df1.columns) * grouped.apply(len).max()

adding mystery column to match with blank created in pd.Series(x.split(','))

newcolumns = newcolumns + ['extra']
ready.columns = newcolumns

giving index a type to help with future merges

ready.index= ready.index.astype('int64')

The line terminator can be changed if its used in any of the column's data.