如何从HDFS FileSystem执行hadoop jar？

Question

I'm always executing job from local File system as following

hadoop jar /home/usr/jar/myjar.jar com.test.TestMain 

How can I execute same command but myjar.jar will be in hdfs ?

Something like

hadoop jar hdfs:///home/usr/jar/myjar.jar com.test.TestMain 

But of course this doesn't work...

Answer 1

hadoop fs -copyToLocal hdfs:///home/usr/jar/myjar.jar /tmp/myjar.jar && hadoop jar /tmp/myjar.jar com.test.TestMain 

Answer 2

Looking at bin/hadoop script for Hadoop, for executing a jar file it uses this condition --

elif [ "$COMMAND" = "jar" ] ; then
  CLASS=org.apache.hadoop.util.RunJar

This says it uses RunJar class to execute the jar. If you see the RunJar class, the arguments supported are --

RunJar jarFile [mainClass] args...

where, jarFile is accessed using the following code inside RunJar Class--

int firstArg = 0;
String fileName = args[firstArg++];
File file = new File(fileName);

Hence, fileName points to jarFile , so fileName in my view cannot be a HDFS path as seen above, unless you are somehow able to mount the HDFS in your file system, so that it becomes accessible by the java File class.