[英]Hadoop WordCount error
I am following the documentation found at this link 我正在关注此链接上找到的文档
https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html#Usage https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html#Usage
When i try to compile for WordCount.java and create a jar, I get the following error 当我尝试为WordCount.java编译并创建一个jar时,出现以下错误
bin/hadoop com.sun.tools.javac.Main WordCount.java
Error: Could not find or load main class com.sun.tools.javac.Main
I have verified my $JAVA_HOME and $HADOOP_CLASSPATH in the hadoop-env.sh file and also verified to see if I have the jdk 我已经在hadoop-env.sh文件中验证了我的$ JAVA_HOME和$ HADOOP_CLASSPATH,还验证了我是否有jdk
Here are the contents from hadoop-env.sh
这是
hadoop-env.sh
中的内容
export JAVA_HOME="/Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/"
.......
.........
for f in $HADOOP_HOME/contrib/capacity-scheduler/*.jar; do
if [ "$HADOOP_CLASSPATH" ]; then
export HADOOP_CLASSPATH="$JAVA_HOME/lib/tools.jar"
else
export HADOOP_CLASSPATH=$f
fi
I am not sure the reason behind error or if I am missing another key configuration? 我不确定出现错误的原因还是缺少其他关键配置?
This doesn't make sense in that loop... Nor does checking the existence of the variable first 这在那个循环中没有意义……也不先检查变量是否存在
if [ "$HADOOP_CLASSPATH" ]; then
export HADOOP_CLASSPATH="$JAVA_HOME/lib/tools.jar"
else
You need to set HADOOP_CLASSPATH="$JAVA_HOME/lib/tools.jar"
, as the documentation says for that class to be found. 您需要按照文档中的说明设置
HADOOP_CLASSPATH="$JAVA_HOME/lib/tools.jar"
。 And that class is only available in the JDK 该类仅在JDK中可用
But, you could just run javac
command to compile code. 但是,您可以只运行
javac
命令来编译代码。 Not sure why the docs have you calling that class. 不知道为什么文档要您调用该类。
How to compile a Hadoop program 如何编译Hadoop程序
$ javac -classpath ${HADOOP_CLASSPATH} -d WordCount/ WordCount.java
To create jar:
创建罐子:
$ jar -cvf WordCount.jar -C WordCount/ .
To run:
跑步:
$ hadoop jar WordCount.jar WordCount input/ output
Suggestion Please use Maven/Gradle to create proper JAR files, and an IDE to write code. 建议请使用Maven / Gradle创建正确的JAR文件,并使用IDE编写代码。
PS Not many people actually write plain MapReduce PS实际写纯MapReduce的人并不多
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.