I have two pandas data frames:
Dataframe 1:
Id TOTAL_CLAIM_COST
023248db-5e9c-42f3-a9fd-4133bee82844 129.16
b851d7d4-813a-4be2-97b6-f16a347628c6 50.00
Dataframe 2:
Id CODE
023248db-5e9c-42f3-a9fd-4133bee82844 3
023248db-5e9c-42f3-a9fd-4133bee82844 1
023248db-5e9c-42f3-a9fd-4133bee82844 2
Desired Output:
Dataframe 3:
Id TOTAL_CLAIM_COST CODE_1 CODE_2 CODE_3
023248db-5e9c-42f3-a9fd-4133bee82844 129.16 3 1 2
b851d7d4-813a-4be2-97b6-f16a347628c6 50.0 NaN NaN Nan
How do I achieve this? I have 50K rows in Dataframe 1 and about 10K rows in Dataframe 2. Secondly, Is there a pythonic way of doing this instead of loops?
========
After playing around, I was able to solve this using for loop. However, it is very ineffective from a performance perspective. Can someone help me know how to replace the for loops in the solution with a pythonic way?
Ineffective solution:
import pandas as pd
data1 = [{'Id': '023248db-5e9c-42f3-a9fd-4133bee82844', 'TOTAL_CLAIM_COST':129.16}, {'Id': 'b851d7d4-813a-4be2-97b6-f16a347628c6', 'TOTAL_CLAIM_COST':50.00}]
df1 = pd.DataFrame(data1)
data2 = [{'Id': '023248db-5e9c-42f3-a9fd-4133bee82844', 'CODE':3}, {'Id': '023248db-5e9c-42f3-a9fd-4133bee82844', 'CODE':1},{'Id': '023248db-5e9c-42f3-a9fd-4133bee82844', 'CODE':2},{'Id': '02eb040d-a1be-4f00-b6cc-eeda3e0b939f', 'CODE':8},{'Id': '02eb040d-a1be-4f00-b6cc-eeda3e0b939f', 'CODE':9}]
df2 = pd.DataFrame(data2)
df2['COUNT'] = df2.groupby('Id')['Id'].transform('count')
num = df2['COUNT'].max()
for i in range(num):
col_name = 'CODE' + '_' + str(i)
df3[col_name] = 0
counter = 0
for index, row in df3.iterrows():
counter = 0
df4 = df2[df2['Id'] == row['Id']]
for index2, row2 in df4.iterrows():
if counter < num:
col_name = '''''' + 'CODE' + '_' + str(counter) + ''''''
df3.at[index,col_name] = row2['CODE']
counter += 1
Output:
Here is a link to another thread that has multiple solutions.
Pandas - Convert columns to new rows after groupby
Here is a solution based on that thread (not sure where you got 70 in your solution).
df=pd.pivot_table(df2,index=['Id'],columns=df2.groupby(['Id']).cumcount().add(1),values=['CODE']).fillna(0)
df.columns=df.columns.map('{0[0]}_{0[1]}'.format)
final_df = pd.concat([df,df1.set_index('Id')],axis=1).fillna(0).reset_index().rename(columns={'index':"Id"})
print(final_df)
Id CODE_1 CODE_2 CODE_3 TOTAL_CLAIM_COST
0 023248db-5e9c-42f3-a9fd-4133bee82844 3.0 1.0 2.0 129.16
1 02eb040d-a1be-4f00-b6cc-eeda3e0b939f 8.0 9.0 0.0 0.00
2 b851d7d4-813a-4be2-97b6-f16a347628c6 0.0 0.0 0.0 50.00
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.