如何计算文字中的单词并追加到词典中？

Question

I'm trying to make a dictionary of frequency of words in a text but for some reason extra characters print out (I'm not sure if this is my text or if it's my code) and it doesn't successfully print out the lines or words that contain the invalid symbol! This is the code I have:

 def parse_documentation(filename):
    filename=open(filename, "r") 
    lines = filename.read(); 
    invalidsymbols=["`","~","!", "@","#","$"]
    for line in lines: 
        for x in invalidsymbols:
            if x in line: 
                print(line) 
                print(x) 
                print(line.replace(x, "")) 
                freq={}
            for word in line:
                count=counter(word)
        freq[word]=count
    return freq

Answer 1

Your code has several flaws. I will not solve all of them but point you in the right direction.

Firstly, read reads the whole file as a string. I don't think that's your intention here. Use readlines() instead to get all lines in the file as a list.

def parse_documentation(filename):
    filename=open(filename, "r") 
    lines = filename.readlines(); # returns a list of all lines in file
    invalidsymbols=["`","~","!", "@","#","$"]
    freq = {} # declare this OUTSIDE of your loop.
    for line in lines:
        for letter in line:
            if letter in invalidsymbols:
                print(letter) 
                line = line.replace(letter, ""))
        print line #this should print the line without invalid symbols.

        words = line.split() # Now get the words.

        for word in line:
            count=counter(word)
            # ... Do your counter stuff here ...

    return freq

Second, I'm highly suspicious of the workings of your counter method. If your intention is to count the number of words, you could adopt this strategy:

1. Check if word is in freq .
2. If it is not in in freq , add it and map it to 1. Otherwise, increment the number that the word was previously mapped to.

This should set you on the right track.

Answer 2

Check this, it might be what you want. BTW, your code is not correct Python code. There are many issues there.

from collections import Counter

def parse_documentation(filename):
    with open(filename,"r") as fin:
        lines = fin.read()
    #for sym in ["`","~","!","@","#","$"]: lines = lines.replace(sym,'')
    lines = lines.translate(None,"`~!@#$")    #thanks to @gnibbler's comment
    freq = Counter(lines.split())
    return freq

text file:

this is a text. text is that. @this #that
$this #!that is those

Results:

Counter({'this': 3, 'is': 3, 'that': 2, 'a': 1, 'that.': 1, 'text': 1, 'text.': 1, 'those': 1})

Answer 3

you might need. line.split(' ') else the for loop will loop through letters.

....
for word in line.split(' '):
    count=counter(word)
...