从给定集中出现的列表中删除所有单词

Question

我正在尝试在Python中高效地完成三个简单的步骤。

我有一个列表列表（字符串）。 让我们称之为L

1. 我想将列表列表展平为单个列表LL。 （我知道如何有效地做到这一点）

2. 从步骤1的列表LL构造频率为1的单词集。让我们将此集称为S。（我也知道如何有效地做到这一点）

3. 从列表L中删除出现在S中的所有单词。

如果您可以建议执行第3步的有效方法，那将是很大的帮助。

Answer 1

对第三步使用简单的list comprehension ：

>>> from collections import Counter
>>> from itertools import chain
>>> L=[['a','b'],['foo','bar'],['spam','eggs'],['b','c'],['spam','bar']]
>>> S=Counter(chain(*L))
>>> S
Counter({'b': 2, 'bar': 2, 'spam': 2, 'a': 1, 'c': 1, 'eggs': 1, 'foo': 1})

>>> [[y for y in x if S[y]!=1] for x in L]
[['b'], ['bar'], ['spam'], ['b'], ['spam', 'bar']]

如果您有R集：

>>> L=[['a','b'],['foo','bar'],['spam','eggs'],['b','c'],['spam','bar']]
>>> R={'a','foo'}
>>> [[y for y in x if y not in R] for x in L]
[['b'], ['bar'], ['spam', 'eggs'], ['b', 'c'], ['spam', 'bar']]

Answer 2

import collections
import operator

LL = reduce(operator.add, L)
counted_L = collections.Counter(LL)
def filter_singles(sublist):
  return [value for value in sublist if counted_L[value] != 1]
no_single_freq_L = [filter_singles(sublist) for sublist in L]

Answer 3

您已经在步骤2中提到过创建一个集合。内置类型set可以使您的步骤3真正易于阅读和理解。

# if you are already working with sets:
LL - S

# or convert to sets
set(LL) - set(S)

快速示例

>>> all_ten = set(range(0,10))
>>> evens = set(range(0,10,2))
>>> odds = all_ten - evens
>>> odds
set([0, 8, 2, 4, 6,])

Answer 4

>>> #Tools Needed
>>> import collections
>>> import itertools
>>> #Just for this example
>>> import keyword
>>> import random
>>> #Now create your example data
>>> L = [random.sample(keyword.kwlist,5) for _ in xrange(5)]
>>> #Flatten the List (Step 1)
>>> LL = itertools.chain(*L)
>>> #Create Word Freq (Step 2)
>>> freq = collections.Counter(LL)
>>> #Remove all words with unit frequency (Step 3)
>>> LL = itertools.takewhile(lambda e:freq[e] > 1,freq)
>>> #Display Your result
>>> list(LL)
['and']
>>> L
[['in', 'del', 'if', 'while', 'print'], ['exec', 'try', 'for', 'if', 'finally'], ['and', 'for', 'if', 'print', 'lambda'], ['as', 'for', 'or', 'return', 'else'], ['and', 'global', 'or', 'while', 'lambda']]
>>>