我如何让hadoop使用与`javac`相同的Java Runtime版本？

Question

I try to see if hadoop's own command can run a map reduce application (from the Hadoop's Map Reduce official document) compiled directly by javac instead of hadoop's own command.

I compile a map reduce application compiled directly by javac

$ javac -classpath "$(~/programs/hadoop/hadoop-3.1.2/bin/hadoop classpath)" -d target/classes/myapp/ src/main/java/myapp/WordCount.java

and create a jar file

$ jar -cvf WordCount.jar -C target/classes/myapp/ .

Why can't I run the jar file by hadoop's own command?

$ export HADOOP_CLASSPATH=$JAVA_HOME/lib/tools.jar
$ rm -r ./output/
$ ~/programs/hadoop/hadoop-3.1.2/bin/hadoop jar WordCount.jar WordCount ./input/ ./output/
Exception in thread "main" java.lang.UnsupportedClassVersionError: WordCount has been compiled by a more recent version of the Java Runtime (class file version 54.0), this version of the Java Runtime only recognizes class file versions up to 52.0
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:232)

How can I make hadoop jar use the same Java Runtime version as javac ?

Thanks.

Answer 1

You compiled it with a newer version of Java (looks like Java SE 10?) than what hadoop is running. Either:

1. Compile your Java program to an older version of java using the -target flag. ie. -target 1.8
2. Have hadoop use the same Java runtime that you compiled your program with. There should be a hadoop-env.sh file that gets called by hadoop to set the JAVA_HOME.

Under your HADOOP_HOME/conf directory please update the hadoop-env.sh file. It has entry to export JAVA_HOME.

Setting to appropriate JAVA_HOME in this file should solve your issue.