I am new to python. I have a simple program to find the number of times a word has been used in a website.
opener = urllib2.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
url = 'https://en.wikipedia.org/wiki/Wolfgang_Amadeus_Mozart'
ourUrl = opener.open(url).read()
soup = BeautifulSoup(ourUrl)
dem = soup.findAll('p') #find paragraphs
word_counts = Counter()
stopwords = frozenset(('A', 'AN', 'THE'))
for i in dem: # loop for each para
words = re.findall(r'\w+', i.text)
cap_words = [word.upper() for word in words if not word.upper() in stopwords]
word_counts.update(cap_words)
print word_counts
Thing is, that this script gives a lot of words which are used only once. How can I update the script so that the word included, has at least 5 word count.
Also how can I arrange the top 5 most common words, into say word1, word2, word3.... etc.
How can i update the script so that the word included, has atleast 5 word count.
You can filter the Counter as follows: filter(lambda x: x[1] > 5, word_counts.iteritems())
filter()
takes a function and an iterable, applies the function to each element of the iterable, and only includes that item in the output if the function returned True
. iteritems()
returns a generator which yields key, value pairs over a dictionary.
how can i arrange the top 5 most common words, into say word1, word2, word3.... etc.
There is a most_common(n)
Counter function. See http://docs.python.org/2/library/collections.html
尝试: print word_counts.most_common(5)
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.