对于给定的字符串列表，计数单词数

Question

I have a given list of string which is:

strings = ["the method of lagrange multipliers is the economists workhorse for solving optimization problems", "the technique is a centerpiece of economic theory but unfortunately its usually taught poorly"]

Now I want to find the no of word in each sentence so that my output will be

{'the': 2, 'method': 1, 'of': 1, 'lagrange': 1, 'multipliers': 1, 'is': 1, 'economists': 1, 'workhorse': 1, 'for': 1, 'solving': 1, 'optimization': 1, 'problems': 1}

{'the': 1, 'technique': 1, 'is': 1, 'a': 1, 'centerpiece': 1, 'of': 1, 'economic': 1, 'theory': 1, 'but': 1, 'unfortunately': 1, 'its': 1, 'usually': 1, 'taught': 1, 'poorly': 1}

My code is as below:

from collections import Counter

dataset = ["the method of lagrange multipliers is the economists workhorse for solving optimization problems",
           "the technique is a centerpiece of economic theory but unfortunately its usually taught poorly"]


for index,row in enumerate(dataset):
    word_frequency = dict(Counter(row.split(" ")))
    
print(word_frequency)

With this i am getting output which is:

{'the': 1, 'technique': 1, 'is': 1, 'a': 1, 'centerpiece': 1, 'of': 1, 'economic': 1, 'theory': 1, 'but': 1, 'unfortunately': 1, 'its': 1, 'usually': 1, 'taught': 1, 'poorly': 1}

Clearly it's only considering the second sentence and counting it but not the first one.

Can anyone help me understand what is wrong in my code?

Answer 1

The word_frequency is being updated with every string in the dataset list.In the end, it is storing the Counter for last string in the dataset.Hence, displaying the the Counter for words in the last string. You can use print(word_frequency) inside the for loop or use a list and append the word_frequency to the list each time and once you are out of the loop just print the list .

from collections import Counter

dataset = ["the method of lagrange multipliers is the economists workhorse for solving optimization problems",
           "the technique is a centerpiece of economic theory but unfortunately its usually taught poorly"]

l = []
for index,row in enumerate(dataset):
    word_frequency = dict(Counter(row.split(" ")))
    l.append(word_frequency)
    
print(l)

Answer 2

Just move the print to be inside your for loop as you're overwriting your calculated word_frequency parameter.

Answer 3

Print inside the loop instead of setting a variable inside the loop (which will get overwritten on each iteration before you print it at the end):

>>> dataset = ["the method of lagrange multipliers is the economists workhorse for solving optimization problems",
...            "the technique is a centerpiece of economic theory but unfortunately its usually taught poorly"]
>>> from collections import Counter
>>> for sentence in dataset:
...     print(dict(Counter(sentence.split())))
...
{'the': 2, 'method': 1, 'of': 1, 'lagrange': 1, 'multipliers': 1, 'is': 1, 'economists': 1, 'workhorse': 1, 'for': 1, 'solving': 1, 'optimization': 1, 'problems': 1}
{'the': 1, 'technique': 1, 'is': 1, 'a': 1, 'centerpiece': 1, 'of': 1, 'economic': 1, 'theory': 1, 'but': 1, 'unfortunately': 1, 'its': 1, 'usually': 1, 'taught': 1, 'poorly': 1}

Answer 4

You're overwriting the word_frequency variable inside the for loop, meaning only the final value from the final count is printed.

You should instead define a Counter outside the for loop, add to the values of that using Counter's add functionality within the for loop, then convert to dict and print at the end:

from collections import Counter

dataset = ["the method of lagrange multipliers is the economists workhorse for solving optimization problems",
           "the technique is a centerpiece of economic theory but unfortunately its usually taught poorly"]

cumulative_counter = Counter()
for index,row in enumerate(dataset):
  cumulative_counter += Counter(row.split(" "))

word_frequency = dict(cumulative_counter)
print(word_frequency)

Outputs:

{'the': 3, 'method': 1, 'of': 2, 'lagrange': 1, 'multipliers': 1, 'is': 2, 'economists': 1, 'workhorse': 1, 'for': 1, 'solving': 1, 'optimization': 1, 'problems': 1, 'technique': 1, 'a': 1, 'centerpiece': 1, 'economic': 1, 'theory': 1, 'but': 1, 'unfortunately': 1, 'its': 1, 'usually': 1, 'taught': 1, 'poorly': 1}