使用Pandas将列重整为行

Question

i have a df

id    name   value
1      abc     10
1      qwe     23
1      zxc     12
2      sdf     10
2      wed     23
2      abc     12
2      mnb     11

i want to reshape this dataframe into:

id    n1    n2    n3    n4
 1    abc   qwe   zxc    0
 2    sdf   wed   abc   mnb

we can see that there are 3 rows for id=1 and 4 rows for id=2. Therefor replace the last column n4=0 incase of such happenings.

This is test dataframe, it may happen that, for a id there might by 1-2 rows too.

it is something like we do in R - dcast. how can we do this in pandas?

Answer 1

Possibly Overkill

f, u = pd.factorize(df.id.values)
b = np.bincount(f)
n, m = u.size, b.max()
c = np.arange(f.size) - np.arange(n).repeat(b) * (m - 1)

v = np.zeros((n, m), dtype=object)
v[f, c] = df.name.values

pd.DataFrame(
    v, pd.Index(u, name='id'),
    ['n{}'.format(i) for i in range(1, m + 1)]
).reset_index()

   id   n1   n2   n3   n4
0   1  abc  qwe  zxc    0
1   2  sdf  wed  abc  mnb

Answer 2

You could go the str route and use some regex replacement and splitting after the groupby .

df.groupby('id').name.apply(lambda x: str(list(x)))\
          .str.replace("[\[\],']", "")\
          .str.split(expand=True).fillna(0)\
          .rename(columns = lambda x: 'n{}'.format(x + 1))

     n1   n2   n3   n4
id                    
1   abc  qwe  zxc    0
2   sdf  wed  abc  mnb

Answer 3

You can use set_index with cumcount for counts per groups for new columns names and reshape by unstack , last rename columns:

df = (df.set_index(['id', df.groupby('id').cumcount()])['name']
       .unstack(fill_value=0)
       .rename(columns = lambda x: 'n{}'.format(x + 1))
       .reset_index())
print (df)
   id   n1   n2   n3   n4
0   1  abc  qwe  zxc    0
1   2  sdf  wed  abc  mnb

Solution with DataFrame constructor, is necessary no NaN values in original data:

df1 = df.groupby('id')['name'].apply(list)
print (df1)
id
1         [abc, qwe, zxc]
2    [sdf, wed, abc, mnb]
Name: name, dtype: object

df = (pd.DataFrame(df1.values.tolist(), index=df1.index)
        .fillna(0)
        .rename(columns = lambda x: 'n{}'.format(x + 1))
        .reset_index())
print (df)
   id   n1   n2   n3   n4
0   1  abc  qwe  zxc    0
1   2  sdf  wed  abc  mnb

And solution with GroupBy.apply and Series constructor:

df1 = (df.groupby('id')['name'].apply(lambda x: pd.Series(x.values, index=range(1,len(x)+1)))
        .unstack(fill_value=0)
        .add_prefix('n')
        .reset_index())
print (df1)

   id   n1   n2   n3   n4
0   1  abc  qwe  zxc    0
1   2  sdf  wed  abc  mnb

Answer 4

By using dfply package it is possible to do like R's dcast .

# for Python3 only
pip install dfply

Use the spread function of dfply .

import pandas as pd
from io import StringIO
from dfply import *

csv = StringIO("""id,name,value
1,abc,10
1,qwe,23
1,zxc,12
2,sdf,10
2,wed,23
2,abc,12
2,mnb,11""")
df = pd.read_csv(csv)

df['sequence'] = df.groupby('id').cumcount()
df = df[["id", "sequence", "name"]] >> spread(X.sequence, X.name)
df = df.set_index("id").fillna(0).rename(columns = lambda x: 'n{}'.format(x + 1)).reset_index()
print(df)
#    id   n1   n2   n3   n4
# 0   1  abc  qwe  zxc    0
# 1   2  sdf  wed  abc  mnb