[英]How to remove duplicates from list of lists which is in pandas data frame
I have below data frame.我有以下数据框。 I want to compare two columns which have list of lists and remove duplicates and then combine both into one.
我想比较具有列表列表的两列并删除重复项,然后将两者合并为一个。 I am trying the below logic but it throws a error "TypeError: unhashable type: 'list'".
我正在尝试以下逻辑,但它会抛出错误“TypeError:unhashable type:'list'”。
data frame:-数据框:-
df = pd.DataFrame({'col1':[[[1452, 5099], [1418, 499]], [[1427, 55099]]],
'col2':[[[1452, 5099], [1417, 490]], [[1317, 55010]]]})
df
col1 col2
0 [[1452, 5099], [1418, 499]] [[1452, 5099], [1417, 490]]
1 [[1427, 55099]] [[1317, 55010]]
res = [list(set(l1).union(l2) - set(l1).intersection(l2)) for l1, l2 in zip(df['col1'].tolist(), df['col2'].tolist())]
print(res)
Error:错误:
TypeError: unhashable type: 'list'
类型错误:无法散列的类型:“列表”
Excepted output:-例外 output:-
res = [[[1452, 5099], [1418, 499],[1417, 490]], [[1427, 55099],[1317, 55010]]]
df['result']=res
print(df)
col1 col2 result
0 [[1452, 5099], [1418, 499]] [[1452, 5099], [1417, 490]] [[1452, 5099], [1418, 499],[1417, 490]
1 [[1427, 55099]] [[1317, 55010]] [[1427, 55099],[1317, 55010]
You need to temporarily convert your lists to tuples to be hashable.您需要暂时将列表转换为元组才能进行哈希处理。
The cleanest is probably to use a helper function:最干净的可能是使用 helper function:
def merge(list_of_lists):
seen = set()
out = []
for l in list_of_lists:
for item in l:
t = tuple(item)
if t not in seen:
out.append(item)
seen.add(t)
return out
df['result'] = [merge(l) for l in zip(df['col1'], df['col2'])]
A more hacky and less readable way would be to use an intermediate dictionary as container:一种更 hacky 且可读性更差的方法是使用中间字典作为容器:
df['result'] = [list({tuple(x): x for l in lst for x in l}.values())
for lst in zip(df['col1'], df['col2'])]
output: output:
col1 col2 result
0 [[1452, 5099], [1418, 499]] [[1452, 5099], [1417, 490]] [[1452, 5099], [1418, 499], [1417, 490]]
1 [[1427, 55099]] [[1317, 55010]] [[1427, 55099], [1317, 55010]]
Add the columns (concat the lists), then map the elements to 2-tuples and use set to remove duplicates:添加列(连接列表),然后将 map 元素添加到 2 元组并使用 set 删除重复项:
df['res'] = df.col1 + df.col2
df.res = [list(set(map(tuple,x)) )for x in df.res]
#df.res:
#0 [(1452, 5099), (1417, 490), (1418, 499)]
#1 [(1317, 55010), (1427, 55099)]
#Name: res, dtype: object
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.