在Windows下运行hadoop作业

Question

I have successfully installed hadoop under windows. (Namenodes and jobtrackers start normally as they should)

Now I'm trying to run some example jobs:

Running the following command (in cygwin)

hadoop jar $HADOOP_INSTALL/hadoop-0.20.2-examples.jar randomwriter random-data

Throws

   Exception in thread "main" java.io.IOException: Error opening job jar: /cygdrive   /c/cygwin/home/Username/hadoop/hadoop-0.20.2-examples.jar
    at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
 Caused by: java.io.FileNotFoundException: \cygdrive\c\cygwin\home\Username\hadoop\hadoop-0.20.2-examples.jar (The system cannot find the path specified)
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(ZipFile.java:127)
    at java.util.jar.JarFile.<init>(JarFile.java:135)
    at java.util.jar.JarFile.<init>(JarFile.java:72)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:88)

Which is probably thrown because of the fact that cygwin can't recognize the CLASSPATH.

The solution (?) to this problem is described here

So, in general i have to execute java programms as

java -classpath `cygpath -wp $CLASSPATH` [arguments]

Now, given that, how exactly should I execute the above hadoop command in conjuction with the last one?

Answer 1

I realize this post is over a year old, but there is no answer marked. You are very close ... this should work without having to edit any configurations.

hadoop jar `cygpath -wp $HADOOP_INSTALL/hadoop-0.20.2-examples.jar` randomwriter random-data

Answer 2

Edit the $HADOOP_HOME/bin/hadoop file and alter the last line which calls the java command according to your cygwin requirement. After that call the example as the initial call.

Last line calling java is

# run it
  exec "$JAVA" -Dproc_$COMMAND $JAVA_HEAP_MAX $HADOOP_OPTS -classpath "$CLASSPATH" $CLASS "$@"