Cloudera Hadoop not running hadoop 1.0.4?

Question

I have written a hadoop 1.0.4 application that runs fine locally in semi-distributed mode. I have also installed Cloudera Hadoop 4 on my cluster. I thought that CDH4 runs hadoop 1.0.4 since it is listed as stable on the hadoop site, but that seems not to be the case. When i run the application on my cluster I get the following errors:

12/11/27 16:14:38 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
12/11/27 16:14:38 INFO input.FileInputFormat: Total input paths to process : 16
12/11/27 16:14:39 INFO mapred.JobClient: Running job: job_201211271520_0004
12/11/27 16:14:40 INFO mapred.JobClient:  map 0% reduce 0%
12/11/27 16:14:50 INFO mapred.JobClient: Task Id : attempt_201211271520_0004_m_000013_0, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
12/11/27 16:14:50 INFO mapred.JobClient: Task Id : attempt_201211271520_0004_m_000000_0, Status : FAILED
... and so on...

Am I right in my assmption that this is because CHD4 is not compatible with hadoop 1.0.4? And if so does anyone know what version is compatible with hadoop 1.0.4? I rather switch cloudera software than rewrite my application.

Answer 1

You are correct; CDH3 uses version 0.20.2, CDH4 uses version 2.0.0. The nomenclature for Hadoop versions is a mess, and I don't pretend to understand it. But it looks like you may be able to use CDH3 based on the following stated in this blog post by Cloudera :

"The CDH3 distribution incorporated the 0.20.2 Apache Hadoop release plus the features of the 0.20.append and 0.20.security branches that collectively are now known as “1.0.” The Apache Hadoop in CDH3 has been the equivalent of the recently announced Apache Hadoop 1.0 for approximately a year now."

If this is the case, I would give CDH3 a try. If it doesn't work, you may just have to look for something besides Cloudera's installation.