[英]Fastest way to remove duplicates in a list without importing libraries and using sets
I was trying to remove duplicates from a list using the following code:我试图使用以下代码从列表中删除重复项:
a = [1,2,3,4,2,6,1,1,5,2]
res = []
[res.append(i) for i in a if i not in res]
But I would like to do this without defining the list I want as an empty list (ie, omit the line res = []
) like:但我想这样做而不将我想要的列表定义为空列表(即省略行
res = []
),例如:
a = [1,2,3,4,2,6,1,1,5,2]
#Either:
res = [i for i in a if i not in res]
#Or:
[i for i in a if i not in 'this list'] # this list is not a string. I meant it as the list being comprehensed
I want to avoid library imports and set()
我想避免库导入和
set()
I think may work for you.我想可能对你有用。 It removes duplicates from the list while keeping the order.
它在保持顺序的同时从列表中删除重复项。
newlist=[i for n,i in enumerate(L) if i not in L[:n]]
For Python3.6+, you can use dict.fromkeys()
:对于 Python3.6+,您可以使用
dict.fromkeys()
:
>>> a = [1, 2, 3, 4, 2, 6, 1, 1, 5, 2]
>>> list(dict.fromkeys(a))
[1, 2, 3, 4, 6, 5]
Create a new dictionary with keys from iterable and values set to value.
创建一个新字典,其中的键来自可迭代对象,值设置为值。
If you are using a lower Python version, you will need to use collections.OrderedDict
to maintain order:如果您使用的是较低的 Python 版本,则需要使用
collections.OrderedDict
来维护订单:
>>> from collections import OrderedDict
>>> a = [1, 2, 3, 4, 2, 6, 1, 1, 5, 2]
>>> list(OrderedDict.fromkeys(a))
[1, 2, 3, 4, 6, 5]
here is a simple benchmark with the proposed solutions,这是建议的解决方案的简单基准,
it shows that dict.fromkeys
will perform the best它表明
dict.fromkeys
将表现最好
from simple_benchmark import BenchmarkBuilder
import random
b = BenchmarkBuilder()
@b.add_function()
def AmitDavidson(a):
return [i for n,i in enumerate(a) if i not in a[:n]]
@b.add_function()
def RoadRunner(a):
return list(dict.fromkeys(a))
@b.add_function()
def DaniMesejo(a):
return list({k: '' for k in a})
@b.add_function()
def rdas(a):
return sorted(list(set(a)), key=lambda x: a.index(x))
@b.add_function()
def unwanted_set(a):
return list(set(a))
@b.add_arguments('List lenght')
def argument_provider():
for exp in range(2, 18):
size = 2**exp
yield size, [random.randint(0, 10) for _ in range(size)]
r = b.run()
r.plot()
Here is a solution using set
that does preserve the order:这是一个使用
set
的解决方案,它确实保留了顺序:
a = [1,2,3,4,2,6,1,1,5,2]
a_uniq = sorted(list(set(a)), key=lambda x: a.index(x))
print(a_uniq)
One-liner, comprehension, O(n)
, that preserves order in Python 3.6+:单行,理解,
O(n)
,保留 Python 3.6+ 中的顺序:
a = [1, 2, 3, 4, 2, 6, 1, 1, 5, 2]
res = list({k: '' for k in a})
print(res)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.