hadoop - java.lang.RuntimeException: java.lang.InstantiationException

Question

I am trying to run a map-reduce program from client side(widows-7), this is the map-reduce class :

Configuration conf =  new Configuration();

    conf.addResource(new Path("C:\\app\\hadoop-2.0.0-cdh4.3.0\\etc\\hadoop\\core-site.xml"));
    conf.addResource(new Path("C:\\app\\hadoop-2.0.0-cdh4.3.0\\etc\\hadoop\\hdfs-site.xml"));

    conf.set("fs.defaultFS", "hdfs://host:8020");
    conf.set("mapred.job.tracker", "host:8021");

    Job job = new Job(conf, "mapRed");
    job.setMapperClass(MapClass.class);
    //job.setInputFormatClass(org.apache.hadoop.mapreduce.lib.input.TextInputFormat.class);
    int numreducers = 1;

    job.setNumReduceTasks(numreducers);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(Text.class);

    Path inp = new Path("/data/test");
    FileInputFormat.addInputPath(job, inp);
    FileOutputFormat.setOutputPath(job, new Path("/data"));
    System.exit(job.waitForCompletion(true) ? 0 : 1);

This program is throwing exception :

    13/12/14 08:27:52 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.RuntimeException: java.lang.InstantiationException
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2307)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:87)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2342)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2324)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:351)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:163)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:335)
    at org.apache.hadoop.fs.Path.getFileSystem(Path.java:194)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(FileInputFormat.java:368)
    at org.gridedge.finalytics.MRTemplate.main(MRTemplate.java:55)
Caused by: java.lang.InstantiationException
    at sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(Unknown Source)
    at java.lang.reflect.Constructor.newInstance(Unknown Source)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:126)
    ... 10 more

I found a simlilar question asked here InstantiationException in hadoop map reduce program and i changed FileInputFormat to TextInputFormat :

 TextInputFormat.addInputPath(job, inp);
 TextOutputFormat.setOutputPath(job, new Path("/data"));

Error remains same and its thrown at line FileInputFormat.addInputPath(job, inp);

Answer 1

The proper configuration is to set TextInputFormat at in setInputFormat method. That line seems to be commented in your code. Please comment it out and then try to use the

FileInputFormat.addInputPaths.

To set the input and output formats, you could see the below lines.

job.setInputFormatClass(TextInputFormat.class); job.setOutputFormatClass(TextOutputFormat.class);

Let me know if you still face any error.