[英]how to delete a list within a list (i.e., a sublist) if any element of that sublist is in another list?
I have a list that contains a number of sublists. 我有一个包含许多子列表的列表。 For example:
例如:
full_list = [[1, 1, 3, 4], [3, 99, 5, 2],[2, 4, 4], [3, 4, 5, 2, 60]]
I also have another list, called omit. 我还有另一个列表,名为省略。 For example:
例如:
omit = [99, 60, 98]
I want to remove the sublists inside of full_list, if any element in that sublist is in the omit list. 如果该子列表中的任何元素在省略列表中,我想删除full_list中的子列表。 For example, I would want the resulting list to be:
例如,我希望结果列表是:
reduced_list = [[1, 1, 3, 4], [2, 4, 4]]
because only these sublists do not have an element that is in the omit list. 因为只有这些子列表没有省略列表中的元素。
I am guessing that there is some easy way to pull this off with a list comprehension but I cannot get it to work. 我猜是有一些简单的方法可以通过列表理解来解决这个问题,但我无法让它工作。 I have tried a bunch of things: For example:
我尝试了很多东西:例如:
reduced_list = [sublist for sublist in full_list if item for sublist not in omit]
Any help would be much appreciated! 任何帮助将非常感激!
ps, The above is a simplified problem. ps,以上是一个简化的问题。 My end goal is to remove sublists from a very long list (eg, 500,000 sublists) of strings if any element (a string) of those sublists is in an "omit" list contain over 2000 strings.
我的最终目标是从非常长的列表(例如,500,000个子列表)中删除子列表,如果这些子列表中的任何元素(字符串)在“省略”列表中包含超过2000个字符串。
Use set
and all()
: 使用
set
和all()
:
>>> omit = {99, 60, 98}
>>> full_list = [[1, 1, 3, 4], [3, 99, 5, 2],[2, 4, 4], [3, 4, 5, 2, 60]]
>>> [item for item in full_list if all(x not in omit for x in item)]
[[1, 1, 3, 4], [2, 4, 4]]
Main difference between this method and @alecxe's(or @Óscar López's) solution is that it all
short-circuits and doesn't create any set or list in the memory while set-intersection returns a new set that contains all items that are common with omit
set and it's length is checked to determine whether any item was common or not.(set-intersection happens internally at C speed so it is faster than normal python loops used in all
) 这种方法与@ alecxe(或@ÓscarLópez)解决方案的主要区别在于它
all
短路并且不会在内存中创建任何集合或列表,而set-intersection返回包含所有常见项目的新集合omit
set并检查它的长度以确定是否有任何项目是常见的。(set-intersection以C速度内部发生,因此它比all
常规python循环更快)
>>> import random
No items intersect: 没有项目相交:
>>> omit = set(random.randrange(1, 10**18) for _ in xrange(100000))
>>> full_list = [[random.randrange(10**19, 10**100) for _ in xrange(100)] for _ in xrange(1000)]
>>> %timeit [item for item in full_list if not omit & set(item)]
10 loops, best of 3: 43.3 ms per loop
>>> %timeit [x for x in full_list if not omit.intersection(x)]
10 loops, best of 3: 28 ms per loop
>>> %timeit [item for item in full_list if all(x not in omit for x in item)]
10 loops, best of 3: 65.3 ms per loop
All items intersect: 所有项目相交:
>>> full_list = [range(10**3) for _ in xrange(1000)]
>>> omit = set(xrange(10**3))
>>> %timeit [item for item in full_list if not omit & set(item)]
1 loops, best of 3: 148 ms per loop
>>> %timeit [x for x in full_list if not omit.intersection(x)]
1 loops, best of 3: 108 ms per loop
>>> %timeit [item for item in full_list if all(x not in omit for x in item)]
100 loops, best of 3: 1.62 ms per loop
Some items intersect: 有些项目相交:
>>> omit = set(xrange(1000, 10000))
>>> full_list = [range(2000) for _ in xrange(1000)]
>>> %timeit [item for item in full_list if not omit & set(item)]
1 loops, best of 3: 282 ms per loop
>>> %timeit [x for x in full_list if not omit.intersection(x)]
1 loops, best of 3: 159 ms per loop
>>> %timeit [item for item in full_list if all(x not in omit for x in item)]
1 loops, best of 3: 227 ms per loop
Try this: 试试这个:
full_list = [[1, 1, 3, 4], [3, 99, 5, 2], [2, 4, 4], [3, 4, 5, 2, 60]]
omit = frozenset([99, 60, 98])
reduced_list = [x for x in full_list if not omit.intersection(x)]
The only change that I made to the input data is that omit
is now a set, for efficiency reasons, as it will allow us to perform a fast intersection (it's frozen because we're not going to modify it), notice that x
doesn't have to be a set. 我对输入数据所做的唯一改变是
omit
现在是一个集合,出于效率原因,因为它将允许我们执行快速交集(它被冻结,因为我们不打算修改它),注意x
不不得不是一套。 Now the reduced_list
variable will contain the expected value: 现在,
reduced_list
变量将包含预期值:
reduced_list
=> [[1, 1, 3, 4], [2, 4, 4]]
Make omit
a set, check for intersection on each step of iteration: omit
一个集合,检查每个迭代步骤的交集:
>>> full_list = [[1, 1, 3, 4], [3, 99, 5, 2],[2, 4, 4], [3, 4, 5, 2, 60]]
>>> omit = [99, 60, 98]
>>> omit = set(omit) # or just omit = {99, 60, 98} for python >= 2.7
>>> [item for item in full_list if not omit & set(item)]
[[1, 1, 3, 4], [2, 4, 4]]
FYI, better use a frozenset
instead of a set as @Óscar López suggested. 仅供参考,@
frozenset
建议,最好使用frozenset
代替套装。 With frozenset
it runs a bit faster: 使用
frozenset
它运行得更快:
import timeit
def omit_it(full_list, omit):
return [item for item in full_list if not omit & set(item)]
print timeit.Timer('omit_it([[1, 1, 3, 4], [3, 99, 5, 2],[2, 4, 4], [3, 4, 5, 2, 60]], {99, 60, 98})',
'from __main__ import omit_it').timeit(10000)
print timeit.Timer('omit_it([[1, 1, 3, 4], [3, 99, 5, 2],[2, 4, 4], [3, 4, 5, 2, 60]], frozenset([99, 60, 98]))',
'from __main__ import omit_it').timeit(10000)
prints: 打印:
0.0334849357605
0.0319349765778
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.