在NLTK中找到两个文本语料库之间的常用词

Question

I am very new to NLTK and am trying to do something.

What would be the best way to find the common words between two bodies of text? Basically, I have one long text file say text1, and another say text2. I want to find the common words that appear in both the files using NLTK.

Is there a direct way to do so? What would be the best approach?

Thanks!

Answer 1

It seems to me that unless you need to do something special with regards to language processing, you don't need NLTK:

words1 = "This is a simple test of set intersection".lower().split()
words2 = "Intersection of sets is easy using Python".lower().split()

intersection = set(words1) & set(words2)

>>> set(['of', 'is', 'intersection'])