Confusion using Mahout Recommender Evaluation

Question

I have a question about using the AverageAbsoluteDifferenceRecommenderEvaluator #evaluate method.

Using a GenericUserBasedRecommender :

new RecommenderBuilder() {
@Override
public Recommender buildRecommender(DataModel model) throws TasteException {
UserSimilarity userSimilarity = new PearsonCorrelationSimilarity(model);
UserNeighborhood userNeighborhood = new NearestNUserNeighborhood(50, userSimilarity, model);
return new GenericUserBasedRecommender(model, userNeighborhood, userSimilarity);
    }
};

the AverageAbsoluteDifferenceRecommenderEvaluator prints this in the beginning time:

12:02:04.000 [pool-1-thread-1] INFO  org.apache.mahout.cf.taste.impl.eval.StatsCallable - Average time per recommendation: 127ms
12:02:04.000 [pool-1-thread-1] INFO  org.apache.mahout.cf.taste.impl.eval.StatsCallable - Approximate memory used: 826MB / 960MB

And getting a recommendation with recommenderBuilder.buildRecommender(model).recommend(101, 10); - takes about 186.091 ms .. thats pretty much something like the average time per recommendation from the AverageAbsoluteDifferenceRecommenderEvaluator

Now, with GenericItemBasedRecommender following happens:

new RecommenderBuilder() {
@Override
public Recommender buildRecommender(DataModel model) throws TasteException {
ItemSimilarity itemSimilarity = new PearsonCorrelationSimilarity(model);
return new GenericItemBasedRecommender(model, itemSimilarity);
    }
};

Evaluation Output:

11:59:19.950 [main] INFO  o.a.m.c.t.i.eval.AbstractDifferenceRecommenderEvaluator - Starting timing of 63493 tasks in 8 threads
11:59:19.979 [pool-1-thread-1] INFO  org.apache.mahout.cf.taste.impl.eval.StatsCallable - Average time per recommendation: 26ms
11:59:19.979 [pool-1-thread-1] INFO  org.apache.mahout.cf.taste.impl.eval.StatsCallable - Approximate memory used: 598MB / 897MB

yea .. it´s every time something like 26ms

But in fact, using this Recommender I have to wait a looong time for an answer -

recommenderBuilder.buildRecommender(model).recommend(101, 10);

GenericItemBasedRecommender —> 49267.09 ms .. or sometimes less ...but never and ever under 100ms !!

I am using the GroupLens 10M data with GroupLensDataModel and the evaluation I am using 0.95 trainingPercentage data and 1.0 evaluationPercentage

With the full evaluationPercentage i will make sure that the recommendations created, takes all items into account.. so I can compare the average recommendation time with a normal recommendation

Why is the average time per recommendation so less by using a GenericItemBasedRecommender with AverageAbsoluteDifferenceRecommenderEvaluator?? But in fact it is not that fast using the method #recommend

I don´t get it .. hope someone clear this out !! Thanks!

Answer 1

This could happen because you have 10,000 items and 72,000 users (7x the number of items).

In terms of performance you should prefer UBCF if you have far less users that item and IBCF otherwise.

Obviously,the quicker it's not necessarily the best in terms of recommendation accuracy.