使用jar命令执行WordCount程序时hadoop中的“无此文件或目录”

Question

I am new to Hadoop and am trying to execute the WordCount Problem.

Things I did so far -

Setting up the Hadoop Single Node cluster referring the below link. http://www.bogotobogo.com/Hadoop/BigData_hadoop_Install_on_ubuntu_single_node_cluster.php

Write the word count problem referring the below link https://kishorer.in/2014/10/22/running-a-wordcount-mapreduce-example-in-hadoop-2-4-1-single-node-cluster-in-ubuntu-14-04-64-bit/

Problem is when I execute the last line to run the program -

hadoop jar wordcount.jar /usr/local/hadoop/input /usr/local/hadoop/output

Following is the error I get - The directory seems to be present The file is also present in the directory with contents

Finally, on a side note I also tried the following directory sturcture in the jar command.

No avail! :/

I would really appreciate if someone could guide me here!

Regards, Paul Alwin

Answer 1

Your first image is using input from the local Hadoop installation directory, /usr

If you want to use that data on your local filesystem, you can specify file:///usr/...

Otherwise, if you're running pseudo distributed mode, HDFS has been setup, and /usr does not exist in HDFS unless you explicitly created it there.

Based on the stacktrace, I believe the error comes from the /app/hadoop/ staging directory path not existing, or the permissions for it are not allowing your current user to run commands against that path

Suggestion: Hortonworks and Cloudera offer pre-built VirtualBox images and lots of tutorial resources. Most companies will have Hadoop from one of those vendors, so it's better to get familiar with that rather than mess around with having to install Hadoop yourself from scratch, in my opinion