Running Hadoop with single node configuration

Question

I have working with hadoop on single node. I am new with hadoop so please bear with me. I have used tutorial " http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/ " and " https://sites.google.com/site/jianpengxu/tutorials/hadoop-setup " to install the hadoop. Command jps shows that hadoop nodes are up and running.

when I try to run the wordcount (for example)

"bin/hadoop jar hadoop-0.20.2-examples.jar wordcount gutenberg gutenberg-output"
i got the error message
"14/01/29 14:22:52 INFO input.FileInputFormat: Total input paths to process : 1
14/01/29 14:22:52 INFO mapred.JobClient: Running job: job_201401241441_0013
14/01/29 14:22:53 INFO mapred.JobClient:  map 0% reduce 0%
14/01/29 14:23:02 INFO mapred.JobClient: Task Id : attempt_201401241441_0013_m_000000_0, Status : FAILED
java.lang.ClassCastException: org.apache.hadoop.mapreduce.lib.input.FileSplit cannot be cast to org.apache.hadoop.mapred.InputSplit
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:323)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)

14/01/29 14:23:08 INFO mapred.JobClient: Task Id : attempt_201401241441_0013_m_000000_1, Status : FAILED
java.lang.ClassCastException: org.apache.hadoop.mapreduce.lib.input.FileSplit cannot be cast to org.apache.hadoop.mapred.InputSplit
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:323)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)

14/01/29 14:23:14 INFO mapred.JobClient: Task Id : attempt_201401241441_0013_m_000000_2, Status : FAILED
java.lang.ClassCastException: org.apache.hadoop.mapreduce.lib.input.FileSplit cannot be cast to org.apache.hadoop.mapred.InputSplit
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:323)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)

14/01/29 14:23:23 INFO mapred.JobClient: Job complete: job_201401241441_0013
14/01/29 14:23:23 INFO mapred.JobClient: Counters: 3
14/01/29 14:23:23 INFO mapred.JobClient:   Job Counters 
14/01/29 14:23:23 INFO mapred.JobClient:     Launched map tasks=4
14/01/29 14:23:23 INFO mapred.JobClient:     Data-local map tasks=4
14/01/29 14:23:23 INFO mapred.JobClient:     Failed map tasks=1"

Can you please give me idea how to solve this issue. I will appreciate your help. Regards,

Answer 1

What hadoop version is installed? Are you using the examples jar that came with the distribution? The errors indicate that hadoop is using the old MR API and the jar is using the new MR API.

Answer 2

The error java.lang.ClassCastException: org.apache.hadoop.mapreduce.lib.input.FileSplit cannot be cast to org.apache.hadoop.mapred.InputSplit occurs also frequently with Hadoop 2.2.0 or 2.3.0 on Windows when trying the attached examples.

In that case, run Hadoop with admin privileges . It might be sufficient to have the right to create symbolic links . With bin\\winutils.exe /? you get the following hint:

The default security settings in Windows disallow non-elevated administrators and all non-administrators from creating symbolic links. The security settings for symbolic links can be changed in the Local Security Policy management console.

Without admin privileges I could only run the examples by using a local mapreduce (in mapred-site.xml , set mapreduce.framework.name to local ).

Answer 3

I had this exact problem. I followed these steps:

1. Navigate to %HADOOP_HOME%/etc/hadoop/
2. Open "mapred-site.xml" and "mapred-site.xml.template" respectively
3. Change "mapreduce.framework.name" to "local"