Pandas拆分列和聚合结果与索引中的重复项

Question

I have the following dataframe:

ID     Type      Value
1        A         311
1        A         223
1        B        1233
2        A         424
2        A         553
3        A          11
3        B           4
3        B           5

I am trying aggreate the "ID" column by splitting the column "Type", such that each ID has its own row and respective columns for Type A and Type B. In the columns "A" and "B" I want to assign the first occurance of each respective value across the rows. If either A or B (or both) are missing I want to assign NaN. To make this idea clear, the following example depicts the result I am looking for:

   ID       A           B
    1      311        1233
    2      424         NaN
    3       11           4

The result keeps the first value that appeared for A (while ignoring the second value for A 223). Since there is no second value for B in ID 1, it just keeps the value 1233. This logic continues for the other ID's.

I've been trying to solve this using .pivot using

df.pivot(columns="Type",values="Value")

which helps me to seperate the Type column, such that I get:

Type      A        B
  0      311      NaN
  1      223      NaN
  2      NaN     1233
  3      11         4

However I am not able to pass the ID column as index, as it gives me the error:

ValueError: Index contains duplicate entries, cannot reshape

Using drop_duplicates on the ID column however results in data loss. Is there any handy way of doing such an operation in pandas?

Answer 1

You need to drop duplicates first before you pivot.

df.drop_duplicates(['ID', 'Type']).pivot('ID', 'Type', 'Value')

Type      A       B
ID                 
1     311.0  1233.0
2     424.0     NaN
3      11.0     4.0

---

Or, use pivot_table with aggfunc='first' :

df.pivot_table(index='ID', columns='Type', values='Value', aggfunc='first')

Type      A       B
ID                 
1     311.0  1233.0
2     424.0     NaN
3      11.0     4.0

---

Performance
This actually depends on your data, and the number of groups. Best is to test it out on your own data.

df_ = df.copy()
df = pd.concat([df_] * 10000, ignore_index=True)

%timeit df.pivot_table(index='ID', columns='Type', values='Value', aggfunc='first')
%timeit df.drop_duplicates(['ID', 'Type']).pivot('ID', 'Type', 'Value')
%timeit df.groupby(['ID', 'Type']).Value.first().unstack(1)

15.2 ms ± 272 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
8.63 ms ± 98 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
8.34 ms ± 246 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Answer 2

Using groupby first

df.groupby(['ID','Type']).Value.first().unstack()
Type      A       B
ID                 
1     311.0  1233.0
2     424.0     NaN
3      11.0     4.0

Or using groupby head with pivot

df.groupby(['ID','Type'],as_index=False).head(1).pivot('ID', 'Type', 'Value')
Type      A       B
ID                 
1     311.0  1233.0
2     424.0     NaN
3      11.0     4.0