Python - 计算存储在列表中的关键字在文本中出现的次数

Question

I have a list KeywordList of 20k+ keywords. I want to check how many keywords in KeywordList appear in multiple, separate text files. I would also like to know the overall frequency of the keywords that appear in the text files. What is the best way to do this?

Answer 1

I would use the bag of words approach: see https://en.wikipedia.org/wiki/Bag-of-words_model

here is an example I had a few years back, extracting word counts from a pandas dataframe:

all_words = df['keywords'].str.split(expand=True).unstack().value_counts()

this gives you a key-value pairing of unique words and their count. Iterate over your files and you should have all of the words with their counts

From there you can convert your keywords and KeywordList to sets and use the intersection function. This will produce you with a set of all keywords in KeywordList