Custom recommender jobs using apache mahout 0.11.2 over hadoop

Question

I am newbie to Apache Mahout. I am using Apache mahout 0.11.2. So to give it a try I created java class called samplereccommender.java as shown below.

package f;
import java.io.File;
import java.io.IOException;
import org.apache.mahout.cf.taste.common.TasteException;
import org.apache.mahout.cf.taste.impl.model.file.FileDataModel;
import  org.apache.mahout.cf.taste.impl.neighborhood.ThresholdUserNeighborhood;
import org.apache.mahout.cf.taste.impl.recommender.GenericUserBasedRecommender;
import org.apache.mahout.cf.taste.impl.similarity.PearsonCorrelationSimilarity;
import org.apache.mahout.cf.taste.model.DataModel;
import org.apache.mahout.cf.taste.neighborhood.UserNeighborhood;
import org.apache.mahout.cf.taste.recommender.RecommendedItem;
import org.apache.mahout.cf.taste.recommender.UserBasedRecommender;
import org.apache.mahout.cf.taste.similarity.ItemSimilarity;
import org.apache.mahout.cf.taste.similarity.UserSimilarity;
import java.util.List;

public class SampleReccommender {
    public static void main(String args[]){
        try{
            DataModel datamodel = new FileDataModel(new File(args[0]));
            //Creating UserSimilarity object.
            UserSimilarity usersimilarity = new PearsonCorrelationSimilarity(datamodel);
           //Creating UserNeighbourHHood object.
            UserNeighborhood userneighborhood = new ThresholdUserNeighborhood(1.0, usersimilarity, datamodel);
          //Create UserRecomender
           UserBasedRecommender recommender = new GenericUserBasedRecommender(datamodel, userneighborhood, usersimilarity);
           List recommendations = (List) recommender.recommend(2, 3);
           System.out.println(recommendations.size());  
           for (int i=0; i< recommendations.size();i++) {
               System.out.println(recommendations.get(i));
           }
        }
catch(Exception e){
    e.printStackTrace();
}
}}

I managed to run same code from command line as
java -cp n.jar f.SampleReccommender n_lib/wishlistdata.txt

Now from what I read on the internet and book "Mahout in action" I understood that same code can be run on hadoop by using following commands. first I need to include my SampleReccommender.java into existing apache-mahout-distribution-0.11.2/mahout-mr-0.11.2-job.jar. So I followed following procedure.

jar uf /Users/rohitjain/Documents/apache-mahout-distribution-0.11.2/mahout-mr-0.11.2-job.jar samplerecommender.jar

then I tried running mahout job using following command

bin/hadoop jar /Users/rohitjain/Documents/apache-mahout-distribution-0.11.2/mahout-mr-0.11.2-job.jar org.apache.mahout.cf.taste.hadoop.item.RecommenderJob -i /input/wishlistdata.txt -o /output/ --recommenderClassName \\ f.SampleRecommender

But it gives me an error as :

Unexpected --recommenderClassName while processing Job-Specific Options:

I tried above command based on the syntax given "mahout in action" book which is as mentioned below

hadoop jar mahout-core-0.5-job.jar \\ org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob \\ -Dmapred.input.dir=input/ua.base.hadoop \\ -Dmapred.output.dir=output \\ --recommenderClassName \\ org.apache.mahout.cf.taste.impl.recommender.slopeone.SlopeOneRecommender

Am I doing anything wrong ? Also tell me whether the code I used for standalone implementation same can be be used for recommenderJobs or it requires all together different implementation?

Answer 1

Mahout in Action is out of date and the code you are using is being deprecated.

These days Mahout runs on more modern compute platforms like Spark. For the latest Mahout Recommender you can start with the Command Line Interface to spark-itemsimilarity and integrated it with Solr or Eleasticsearch. Or you can pickup a fully integrated end-to-end solution linked below:

• Building a recommender with Mahout: http://mahout.apache.org/users/algorithms/recommender-overview.html
• Mahout spark-itemsimilarity: http://mahout.apache.org/users/recommender/intro-cooccurrence-spark.html
• Universal Recommender from ActionML: https://github.com/actionml/template-scala-parallel-universal-recommendation The UR is built on PredictionIO ML Framework here: https://github.com/actionml/PredictionIO