如何计算此python列表的频率？

Question

I have a list of python list like this:

base_list (About 3,000,000 sub lists):

[
   ['Hello','World','Lucy','Lily'],
   ['Hello','Smith','Simpson','Bart'],
   ....
]

Now i get a small list:

small_list:

['Hello','World']

Now, i need to find out how many times the small_list appears in the base_list.

Appear means this : [1,3] is appears in [1,2,3,4,5] .

UPDATE

I've tried this:

1.Change the base_list into a list of set.

2.Then, change the small_list into a set too:

def get_original_freq(self, actors):
    count = 0
    s = set(actors)
    for row in self.orignal_rows:
      if s.issubset(row):
        count += 1
    return count

But the code runs really slow, about 1000 records have been checked per second.

Answer 1

My first reaction is to answer with a silly (albeit working) answer:

def sublistCount(listA, listB):
    if not len(listB):
        return 0
    conditions = ["%s in a" % repr(b) for b in listB]
    comprehension = '[a for a in listA if %s]' % ' and '.join(conditions)
    return len(eval(comprehension))

where listA is the list of lists and listB is the sublist.

This is actually pretty fast, even when working with lists of strings. I ran through a list of 3,000,000 lists of strings in about 1-2 seconds.

I called it silly because it's using the eval() function to create code on the fly. If you're not sure what your input will be, this could be potentially dangerous. This solution is the bassoon of the orchestra of possible solutions: it's funny, it works, but just one bad note or squeak makes it all bad.

However, my favorite of the potential solutions is this:

def sublistCount(listA, listB):
    b = set(listB)
    matches = [a for a in listA if b.issubset(a)]
    return len(matches)

This is safer, much cleaner, and performs nearly as well as the first solution (for 3,000,000 records).

Answer 2

I find out the Inverted Index helps me out:

1.Make the base_list become a inverted index:

{
    'Hello': [1,5,10,8000]
    'World': [1,2,3,5,9]
    ...
}

2.When i need to count the ['Hello','World']'s count number of appearances. I just find the two inverted index of them and count their common documents.