[英]How this input and output for Hadoop works?
I just setup my single node Hadoop cluster on my local machine using this tutorial - 我只是使用本教程在我的本地机器上设置我的单节点Hadoop集群 -
http://ac31004.blogspot.com/2013/10/installing-hadoop-2-on-mac_29.html http://ac31004.blogspot.com/2013/10/installing-hadoop-2-on-mac_29.html
In this tutorial I followed this following instructions : try a ls hadoop fs -ls 在本教程中,我按照以下说明操作:尝试使用ls hadoop fs -ls
if you get 如果你得到
ls: `.': No such file or directory ls:`。':没有这样的文件或目录
then there is no home directory in the hadoop file system. 那么hadoop文件系统中没有主目录。 So
所以
hadoop fs -mkdir /user
hadoop fs -mkdir /user/<username>
where is the name you are logged onto the machine with. 您登录到计算机的名称在哪里。
now change to $HADOOP_INSTALL directory and upload a file 现在更改为$ HADOOP_INSTALL目录并上传文件
hadoop fs -put LICENSE.txt
When I used this following command : 当我使用以下命令时:
hadoop jar ./hadoop-mapreduce-examples-2.2.0 wordcount LICENSE.txt out
It worked but I am not able to find the output folder. 它工作,但我无法找到输出文件夹。 So to debug I tried to use another output folder and used :
所以要调试我试图使用另一个输出文件夹并使用:
hadoop jar ./hadoop-mapreduce-examples-2.2.0 wordcount LICENSE.txt /Applications/hadoop-2.2.0/out2
This worked fine, but the problem is I don`t seethe directory out2 in the application folder. 这工作正常,但问题是我没有在应用程序文件夹中找到目录out2。
Then I created a directory 'in' in the /Applications/hadoop-2.2.0 and put README.txt in it and tried to use following command : 然后我在/Applications/hadoop-2.2.0中创建了一个目录'in'并将README.txt放入其中并尝试使用以下命令:
hadoop jar ./hadoop-mapreduce-examples-2.2.0.jar wordcount /Applications/hadoop-2.2.0/in/README.txt /Applications/hadoop-2.2.0/out3
Now I got an error: 现在我收到一个错误:
ERROR security.UserGroupInformation: PriviledgedActionException as:chinmay (auth:SIMPLE) cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://localhost:9000/Applications/hadoop-2.2.0/in/README.txt
错误security.UserGroupInformation:PriviledgedActionException as:chinmay(auth:SIMPLE)cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException:输入路径不存在:hdfs:// localhost:9000 / Applications / hadoop-2.2。 0 /中/ README.txt中
The input and output folders are also on HDFS. 输入和输出文件夹也在HDFS上。 Create them there, not on the local filesystem.
在那里创建它们,而不是在本地文件系统上。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.