无法找到或加载主类org.apache.giraph.yarn.GiraphApplicationMaster

Question

I am attempting ito get Giraph running on a YARN cluster, (Hadoop 2.5.2) but am I'm stuck at this error:

Could not find or load main class org.apache.giraph.yarn.GiraphApplicationMaster

I've tried everything I can find in previous messages on this topic, to no avail. My command line is like this:

hadoop jar /home/prhodes/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.5.2-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/prhodes/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/prhodes/giraph_output/shortestpaths -w 4 -yj /home/prhodes/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.5.2-jar-with-dependencies.jar,/home/prhodes/giraph/giraph-core/target/giraph-1.2.0-SNAPSHOT-for-hadoop-2.5.2-jar-with-dependencies.jar

I've checked that those jars are in those locations on each node in the cluster. Also, I've verified that the jar file does contain the GiraphApplicationMaster.class. And yet it still complains about not finding the GiraphApplicationMaster.

Any ideas what else might be causing the problem, or any workarounds? I thought about distributing the Giraph jars to each node and physically putting them on the Hadoop Classpath, but the Maven build builds fat jars by default, which would probably cause issues doing that. Any other suggestions or ideas?

Answer 1

I resolved this issue with commands:

cp $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar $HADOOP_HOME/share/hadoop/yarn/lib/

cp $GIRAPH_HOME/giraph-examples/target/ $HADOOP_HOME/share/hadoop/yarn/lib/

then, run your job