简体繁体 English

如何找到我的 word2vec model 的准确度、精确度、召回率和 f1 分数？

[英]How to find accuracy, precision, recall, f1 score for my word2vec model?

原文 2020-07-28 13:08:48 0 1 python/ cross-validation/ word2vec/ evaluation

I am working on a project to find similarity among products.我正在开展一个项目来寻找产品之间的相似性。 The model splits the excel data sheet into 90% training / 10% validation. model 将 excel 数据表拆分为 90% 训练/10% 验证。 When I check manually for validation the model works pretty well.当我手动检查验证时，model 工作得很好。 But I am having trouble with the evaluation process.但是我在评估过程中遇到了麻烦。 How should I find accuracy, precision, recall and F1 score to understand how well my model works?我应该如何找到准确度、精确度、召回率和 F1 分数来了解我的 model 的效果如何？

I am very new to machine learning, still learning, please give me some clues where to start.我是机器学习的新手，还在学习，请给我一些从哪里开始的线索。

1 个解决方案

Word2vec is an algorithm that's considered 'unsupervised' – it's not trained using specified 'correct' answers, but rather learns from the patterns in any data. Word2vec 是一种被认为是“无监督”的算法——它不是使用指定的“正确”答案进行训练，而是从任何数据中的模式中学习。 As a result, there's no native-to-word2vec idea of 'accuracy', 'precision', etcetera – those concepts only have meaning in relation to a set of desired answers.因此，没有 word2vec 原生的“准确性”、“精确度”等概念——这些概念仅在与一组所需答案相关时才有意义。

So to calculate those values, you have to use those word-vectors in some other downstream task, and devise your own evaluation for that downstream task.因此，要计算这些值，您必须在其他下游任务中使用这些词向量，并且 devise 您自己对该下游任务的评估。 Then you can calculate accuracy & other values for that whole system (including the word2vec step).然后您可以计算整个系统的准确性和其他值（包括 word2vec 步骤）。 This may include applying your judgement, or that of other reviewers, about what the result "should" be in certain cases.这可能包括应用您或其他审阅者的判断，判断在某些情况下“应该”的结果是什么。

Without any examples of your data, it's not yet clear what your Word2Vec model is doing, and how products are represented in it.没有任何数据示例，尚不清楚您的 Word2Vec model 在做什么，以及产品在其中的表示方式。 (What's the individual items in the customers_train list you've created? Where do product names/identifiers come in? What kinds of similarity-questions or end-user operations do you need to be performing?) （您创建的customers_train列表中的各个项目是什么？产品名称/标识符从何而来？您需要执行哪些相似性问题或最终用户操作？）