[英]pandas how to remove duplicated value
name date
a [01-01,01-01,01-03]
b [02-01.03-03.03-03,03-05]
.. ..
.. ..
this is my dataframe 这是我的数据框
data was having a duplicated id and date so i make groupby id 数据具有重复的ID和日期,所以我使groupby ID
df=DataFrame(data)
uid=df['uid']
dt=df['dt']
df1=pd.Series(uid,name='uid')
df3=pd.Series(dt,name='dt')
df=pd.concat([df1,df3], axis=1,ignore_index=True)
df.groupby(uid, as_index=False).agg(lambda x: x.tolist())
my desired output is like this 我想要的输出是这样的
name date
a [01-01,01-03]
b [02-01,03-03,03-05]
.. ..
.. ..
尝试:
df.date = df.date.apply(lambda x: list(set(x)))
if you want to remove duplicates and also sort them based on initial order. 如果您要删除重复项并根据初始顺序对其进行排序。 see example below: 请参见下面的示例:
df = pd.DataFrame.from_dict({'name':['a','b'], 'date': [['01-01','01-01','01-03'],['02-01','03-03','03-03','03-05']]})
print 'before removing duplicates'
print df
print 'after removing duplicates and sorting based on initial order'
df['date'] = df['date'].apply(lambda x: sorted(list(set(x)), key = x.index))
print df
results in 结果是
before removing duplicates
date name
0 [01-01, 01-01, 01-03] a
1 [02-01, 03-03, 03-03, 03-05] b
after removing duplicates and sorting based on initial order
date name
0 [01-01, 01-03] a
1 [02-01, 03-03, 03-05] b
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.