[英]Mahout recommender evaluation - how to use a fixed test set
I'm writing a recommender system evaluator with Apache Mahout, using a train.csv training set and the Precision metric. 我正在使用train.csv训练集和Precision指标使用Apache Mahout编写推荐系统评估程序。 My question is: it's possible to use a fixed test set, not generated by the evaluator?
我的问题是:是否可以使用固定的测试集,而不是由评估员生成?
To be more specific, I have a test.csv file that contains a list of UserIds and for these I want to provide recommendations and evaluate the results with the Precision metric, only for this fixed set of users that never changes. 更具体地说,我有一个test.csv文件,其中包含UserId列表,为此,我想提供建议并使用Precision指标评估结果,仅适用于从未改变过的固定用户集。 Their ratings are in the file train.csv, I use it to train the algorithm and it contains also all the other user's ratings.
它们的等级在train.csv文件中,我用它来训练算法,它还包含所有其他用户的等级。
I post also the code where I want to add this feature: 我还在要添加此功能的代码中发布了代码:
RandomUtils.useTestSeed();
DataModel model = new FileDataModel(new File("files/train.csv"));
RecommenderIRStatsEvaluator evaluator = new GenericRecommenderIRStatsEvaluator();
RecommenderBuilder recommenderBuilder = new RecommenderBuilder() {
public Recommender buildRecommender(DataModel model) throws TasteException {
//Here I build my recommender system
//return ...
}
};
IRStatistics stats = evaluator.evaluate(recommenderBuilder, null, model, null, 5,
4/*relevance Threshold*/, 1);
System.out.println(stats.getPrecision());
So you want cross-validation gold standard test data which you have. 因此,您需要具有的交叉验证金标准测试数据。 It is split into train and test.
它分为训练和测试。 You would like a repeatable test.
您想要一个可重复的测试。 This make a lot of sense.
这很有道理。
The Mahout evaluator does the split for you based on a random picking of test and training data from what you pass in. If you pass in a fixed RNG seed the evaluator will pick the exact same test and training set. Mahout评估程序将根据您传入的随机选择的测试和训练数据为您进行拆分。如果传入固定的RNG种子,则评估程序将选择完全相同的测试和训练集。 This isn't exactly what you asked but it is one way to get repeatable CV tests.
这并不是您所要的,但这是获得可重复的CV测试的一种方法。
Otherwise you will need to hack the Evaluator to use your pre-calculated test/training sets. 否则,您将需要破解评估程序以使用您预先计算的测试/培训集。
The precision metric I use is mean average precision (MAP) at some number of recommendations, like the number you will calculate or show in the UI. 我使用的精度指标是某些建议下的平均平均精度(MAP),例如您将计算或显示在UI中的数字。 This is not built into the Mahout Evaluator.
Mahout评估程序未内置该功能。
To do all this right you'd hack the Evaluator. 要做到这一切,您需要破解评估程序。
BTW I wouldn't use that recommender unless absolute simplicity is the highest design criteria. 顺便说一句,除非绝对简单是最高的设计标准,否则我不会使用该推荐程序。 The newest Mahout recommenders build models that are queried using a search engine like Solr or Elasticsearch.
最新的Mahout推荐器可使用Solr或Elasticsearch等搜索引擎来构建要查询的模型。 These are incredibly flexible and scalable.
这些都是非常灵活和可扩展的。
The new way described here: http://mahout.apache.org/users/recommender/intro-cooccurrence-spark.html Some blog posts about this method here: http://occamsmachete.com/ml/ 此处描述的新方法: http : //mahout.apache.org/users/recommender/intro-cooccurrence-spark.html有关此方法的一些博客文章位于: http : //occamsmachete.com/ml/
With this method you'd train on your train.csv and use the user history in test.csv to make queries. 使用这种方法,您可以在train.csv上进行训练,并使用test.csv中的用户历史记录进行查询。 Calculate the precision of all queries using MAP.
使用MAP计算所有查询的精度。 The new method uses a search engine for the queries so you also have a service that is scalable.
新方法使用搜索引擎进行查询,因此您还具有可扩展的服务。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.