獲取IndexError：計算歐幾里得距離時列表索引超出范圍

Question

我正在嘗試應用https://towardsdatascience.com/3-basic-distance-measurement-in-text-mining-5852becff1d7提供的代碼。 當我將它與我自己的數據一起使用時，我似乎訪問了不存在的列表的一部分，並且無法確定我在哪里犯了這個錯誤：

File "D:/Proj/projects/eucledian_1/nltk_headline_1.py", line 190, in eucledian
    score = sklearn.metrics.pairwise.euclidean_distances([transformed_results[i]], [transformed_results[0]])[0][0]
IndexError: list index out of range

我認為我沒有超過only_event或“transformed_results”的長度。 這是我擁有的代碼：

def eucledian(self):
        print('inside eucledian', only_event) 
        for i, news_headline in enumerate(only_event): # range(len(only_event))
            print('*******', only_event[i])
            print('this is transformed results: ', transformed_results[i]) # prints
            score = sklearn.metrics.pairwise.euclidean_distances([transformed_results[i]], [transformed_results[0]])[0][0]
            print('-----', only_event[i]) # prints
            print('Score: %.2f, Comparing Sentence: %s' % (score, news_headline)) # prints

我能夠從 DB 中讀取並存儲在列表only_event (length = 2) 中的數據如下所示： ['Perhaps this code is incomplete or mistyped in some way.', 'Use one of the following methods:\n* Ensure that the power is turned on.\n* Only concatenate a user-supplied value into a query, or if it must be a Boolean or numeric type.\n'] 。

打印語句給出 output，但調用 euclidean_distances 的行拋出IndexError: list index out of range錯誤。 transformed_results的結果（長度 = 1）如下所示：

[array([329.,   2.,  57.,  44.,  44.,  44.,  88.,  57.,  44.,  44.,  44.,
        57.,  13.,  13.,  88.,   1.,   2.,  13., 136.,  13.,  13.,  13.,
       220.,  44.,  44.,  44.,  88.,  88.,  44.,  44.,  89.,   2.,  13.,
        88.,  13.,  44., 132.,  26.,   4.,   4., 132.,  44.,   1.,  13.,
        48.,  27.,  88., 132.,  88.,  44.,  44., 132.,  13.,   4.,  13.,
        44.,  13., 158.,  15.,  13., 162.,   4.,  44.,  44.,  26.,  13.,
         1.,  44.,   1.,  57.,  13.,   1.,  44.,  44.,  45.,  44.,  44.,
         4.,  13.,  44.,   1.,  13.,  44.,  44.,  44.,  44., 336.,  44.,
        51.,   2., 235.,  13., 132., 132.,  70.,  26.,  44.,  13.,  13.,
        13.,  44.,   4.,   1.,  57.,  44.,  44.,   2.,  44.])]

提前感謝您瀏覽此內容

更新為包含可重現的代碼@dzang

import numpy as np
import sklearn.preprocessing
import sklearn.metrics

token_event_obj = ['perhaps', 'this', 'code', 'is', 'incomplete', 'or', 'mistyped', 'in', 'some', 'way', 'use', 'one', 'of', 'the', 'following', 'methodsn', 'use', 'a', 'querypreparation', 'api', 'to', 'safely', 'construct', 'the', 'sql', 'query', 'containing', 'usersupplied', 'valuesn', 'only', 'concatenate', 'a', 'usersupplied', 'value', 'into', 'a', 'query', 'if', 'it', 'has', 'been', 'checked', 'against', 'a', 'whitelist', 'of', 'safe', 'string', 'values', 'or', 'if', 'it', 'must', 'be', 'a', 'boolean', 'or', 'numeric', 'typen']
only_event = ['Perhaps this code is incomplete or mistyped in some way.', 'Use one of the following methods:\n* Use a query-preparation API to safely construct the SQL query containing user-supplied values.\n* Only concatenate a user-supplied value into a query if it has been checked against a whitelist of safe string values, or if it must be a Boolean or numeric type.\n']

def transform(headlines):
    print('inside transform', headlines)
    tokens = [w for s in headlines for w in s]
    print()
    print('All Tokens:')
    print(tokens)

    results = []
    label_enc = sklearn.preprocessing.LabelEncoder()
    onehot_enc = sklearn.preprocessing.OneHotEncoder()

    encoded_all_tokens = label_enc.fit_transform(list(set(tokens)))
    encoded_all_tokens = encoded_all_tokens.reshape(len(encoded_all_tokens), 1)

    onehot_enc.fit(encoded_all_tokens)

    for headline_tokens in headlines:
        print()
        print(headline_tokens)
        print('Original Input:', headline_tokens)

        encoded_words = label_enc.transform(headline_tokens)
        print('Encoded by Label Encoder:', encoded_words)

        encoded_words = onehot_enc.transform(encoded_words.reshape(len(encoded_words), 1))
        print('Encoded by OneHot Encoder:')
        # print(encoded_words)

        results.append(np.sum(encoded_words.toarray(), axis=0))
        print('Transform results:', results)

    return results


def eucledian():
        print('inside eucledian', len(only_event))
        for i, news_headline in enumerate(only_event): # range(len(only_event))
            print('*******', only_event[i])
            print('this is transformed results: ', transformed_results)
            # print('len', len(sklearn.metrics.pairwise.euclidean_distances([transformed_results[i]], [transformed_results[0]])[0]))
            print(type(transformed_results), len(transformed_results))
            score = sklearn.metrics.pairwise.euclidean_distances([transformed_results[i]], [transformed_results[0]])[0]
            print('-----', only_event[i])
            print('Score: %.2f, Comparing Sentence: %s' % (score, news_headline))


transformed_results = transform([token_event_obj])
eucledian()

Answer 1

您提供的示例中的錯誤在於， transformed_results是一個包含一個元素的列表，其中包含標記化的句子 1。

only_event雖然有 2 個句子，但您正在使用它來提供i 。 所以i將是0和1 。 當i為1時， transformed_results[i]會引發錯誤。

如果您在only_event中標記兩個句子，例如：

headlines = [''.join([c for c in s.replace('\n', '').lower() if c not in ['.', '*', ':', '-']]).split() for s in only_event]

這使：

[['也許', 'this', 'code', 'is', 'incomplete', 'or', 'mistyped', 'in', 'some', 'way'], ['use', 'one ', 'of', 'the', 'following', 'methods', 'use', 'a', 'querypreparation', 'api', 'to', 'safely', 'construct', 'the', 'sql'、'query'、'包含'、'usersupplied'、'values'、'only'、'concatenate'、'a'、'usersupplied'、'value'、'into'、'a'、'query ', 'if', 'it', 'has', 'been', 'checked', 'against', 'a', 'whitelist', 'of', 'safe', 'string', 'values,' , 'or', 'if', 'it', 'must', 'be', 'a', 'boolean', 'or', 'numeric', 'type']]

然后transformed_results的長度也將為2。

您將比較兩個句子的歐幾里得距離，包括參考句子與其本身。