Spark MLlib - 如何验证隐式反馈协同过滤器

Question

I'm programming it with Scala, but the language doesn't matter here.

The input to implicit feedback collaborative filter (ALS.trainImplicit) are, in this case, views of a product:

• Rating("user1", "product1", 21.0) //Means that user1 has viewed the details of product1 21 times
• Rating("user2", "product1", 4.0)
• Rating("user3", "product2", 7.0)

But the output (MatrixFactorizationModel.recommendProductsForUsers) are like:

• Rating("user1", "product1", 0.78)
• Rating("user2", "product1", 0.63)

Values 0.78 and 0.64 in the output looks like something normalized between 0 and 1, but values in the input were 21,4,7,etc.

I don't think that in this case it has any sense to calculate MSE (mean squared error) between the input and output as we can do when we are using collaborative filters with explicit feedback.

So, the question is, how to validate collaborative filter when using implicit feedback?

Answer 1

Important KPI for Implicit Feedback Validation are eg Accuracy, Coverage and many others. It really depends on the use case (how many product do you want to show? How many products do you to offer?) and the goal you want to achieve.

When I build implicit feedback ALS model I always calculate this 2 KPI. Models with a very good accuarcy tend to cover a smaller amount of the available products to offer. Alsways calculate the coverage and decide from there.

Take a closer look to this post: https://stats.stackexchange.com/questions/226825/what-metric-should-i-use-for-assessing-implicit-matrix-factorization-recommender

and this Spark Library: https://github.com/jongwook/spark-ranking-metrics