NLTK: TypeError: unhashable type: 'list'

Question

i'm following the original code for bleu scoring as below:

from nltk.translate.bleu_score import sentence_bleu

reference = [['this', 'is', 'a', 'test'], ['this', 'is' 'test']]
candidate = ['this', 'is', 'a', 'test']
score = sentence_bleu(reference, candidate)
print(score)

and the code working fine. but i'm trying to change the reference and candidate by importing the csv file as the following code:

import nltk
import csv
import itertools
from nltk.translate.bleu_score import sentence_bleu

print("Opening references file...")
with open('bleu-ref.csv', newline='') as csvfile:
     spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|')
     sentences = []
     for row in spamreader:
         # print(', '.join(row))
         sentences.append(' '.join(row))

sent = [[i] for i in sentences]

reference = []
for i in range(len(sent)):
    sent[i]
    chink = []
    for j in sent[i]:
        chink = chink + nltk.word_tokenize(j)

    reference.append(chink)

print("Opening candidates file...")
with open('bleu-can.csv', newline='') as csvfile:
     spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|')
     sentences_can = []
     for row in spamreader:
         # print(', '.join(row))
         sentences_can.append(' '.join(row))

sent_can = [[i] for i in sentences_can]

candidate = []
for i in range(len(sent_can)):
    sent_can[i]
    chink_can = []
    for j in sent_can[i]:
        chink_can = chink_can + nltk.word_tokenize(j)

    candidate.append(chink_can)

score = sentence_bleu(reference, candidate)

but it encountered an error:

    Traceback (most recent call last):
  File "nltk-bleu-score.py", line 56, in <module>
    score = sentence_bleu(reference, candidate)
  File "C:\Users\Fachri\Anaconda3\lib\site-packages\nltk\translate\bleu_score.py", line 89, in sentence_bleu
    emulate_multibleu)
  File "C:\Users\Fachri\Anaconda3\lib\site-packages\nltk\translate\bleu_score.py", line 162, in corpus_bleu
    p_i = modified_precision(references, hypothesis, i)
  File "C:\Users\Fachri\Anaconda3\lib\site-packages\nltk\translate\bleu_score.py", line 292, in modified_precision
    counts = Counter(ngrams(hypothesis, n)) if len(hypothesis) >= n else Counter()
  File "C:\Users\Fachri\Anaconda3\lib\collections\__init__.py", line 535, in __init__
    self.update(*args, **kwds)
  File "C:\Users\Fachri\Anaconda3\lib\collections\__init__.py", line 622, in update
    _count_elements(self, iterable)
TypeError: unhashable type: 'list'

then, i check the type of reference and candidate , both from the original code and the modified, it return the same type list

i confused what's make those list different.

the list of reference and `candidate' dispaled as below

Opening references file...
[['two', 'airplanes', 'are', 'waiting', 'on', 'the', 'tarmac'], ['Two', 'airplanes', 'parked', 'at', 'the', 'airport', '.']]

Opening candidates file...
[['An', 'airplane', 'sitting', 'on', 'the', 'tarmac', 'at', 'an', 'airport', 'with', 'another', 'plane', 'in', 'the', 'background', '.']]

Answer 1

The expected type for hypothesis is list(str) , from documentation :

:type hypothesis: list(str)

candidate is a list(list(str)) , you can compute the bleu_score like this:

from nltk.translate.bleu_score import sentence_bleu

references = [['two', 'passenger', 'planes', 'on', 'a', 'grassy', 'plain'],
              ['An', 'airplane', 'sitting', 'on', 'the', 'tarmac', 'at', 'an', 'airport', 'with', 'another', 'plane',
               'in', 'the', 'background', '.'],
              ['A', 'white', 'an', 'blue', 'airplane', 'parked', 'at', 'the', 'airport', 'near', 'another', 'small',
               'plane', '.'], ['Blue', 'and', 'white', 'airplane', 'parked', '.'],
              ['two', 'airplanes', 'are', 'waiting', 'on', 'the', 'tarmac'],
              ['Two', 'airplanes', 'parked', 'at', 'the', 'airport', '.'],
              ['A', 'passenger', 'aircraft', 'with', 'landing', 'gear', 'down', '.'],
              ['A', 'passenger', 'jet', 'flies', 'through', 'the', 'air', '.'],
              ['A', 'passenger', 'plane', 'fly', 'through', 'the', 'sky', '.'],
              ['The', 'Austrian', 'plane', 'soars', 'in', 'the', 'sky', '.']]

candidates = [
    ['An', 'airplane', 'sitting', 'on', 'the', 'tarmac', 'at', 'an', 'airport', 'with', 'another', 'plane', 'in', 'the',
     'background', '.'], ['A', 'passenger', 'jet', 'flies', 'through', 'the', 'air', '.']]

for candidate in candidates:
    print(sentence_bleu(references, candidate))

Output

1.0
1.0