Remove elements from nested list - Python

Question

data = [['A', 'B', 'C', 'D'],
        ['E', 'F', 'G'],
        ['I', 'J'],
        ['A', 'B', 'C', 'E', 'F']]

I would like to remove unpopular elements (appearing only once) from the lists. So the results should look like this:

data = [['A', 'B', 'C'],
        ['E', 'F'],
        ['A', 'B', 'C', 'E', 'F']]

I was able to count the frequency of each element using the following codes:

from collections import Counter
Counter(x for sublist in data for x in sublist)

#output
Counter({'A': 2, 'C': 2, 'B': 2, 'E': 2, 'F': 2, 'D': 1, 'G': 1, 'I': 1, 'J': 1})

However, I am not sure how to use this count information to remove unpopular elements from the list. Any help?

Answer 1

Generate the new list based on the frequency information.

The following code uses nested list comprehension to do that:

from collections import Counter
freq = Counter(x for sublist in data for x in sublist)
data = [[x for x in row if freq[x] > 1] for row in data]  # Remove non-popular item
data = [row for row in data if row]  # Remove empty rows

# data => [['A', 'B', 'C'], ['E', 'F'], ['A', 'B', 'C', 'E', 'F']]

Answer 2

The complexity is similar. Just use map and filter function to make the code more pythonic.

from collections import Counter
data = [['A', 'B', 'C', 'D'],
       ['E', 'F', 'G'],
       ['I', 'J'],
       ['A', 'B', 'C', 'E', 'F']]
counter = Counter({'A': 2, 'C': 2, 'B': 2, 'E': 2, 'F': 2, 'D': 1, 'G': 1, 'I': 1, 'J': 1})


result = map(lambda row: filter(lambda x: counter.get(x) > 1, row), data)
print result