Copying files into HDFS Hadoop

Question

I am currently working on a project for one of my lectures at the university. The task is to download a book from https://www.gutenberg.org/ and copy it into HDFS. I've tried using put <localSrc> <dest> but it didnt work at all. This is how my code looks in Terminal at the moment:

[cloudera@quickstart ~]$ put <pg16328.txt> <documents>

bash: syntax error near unexpected token `<'

Any help is appreciated. Thanks in advance.

UPDATE 30.05.2017: I haved used following link https://www.cloudera.com/downloads/quickstart_vms/5-10.html to install Hadoop and did not configure anything at all. Only thing I did was to absolve the tutorial Getting started .

Answer 1

It should just be:

hdfs fs -copyFromLocal pg16328.txt /HDFS/path

I'm not familiar with the put command, but have you tried it without the <> s?

Answer 2

If you have successfully extracted and configured Hadoop , then

you should be in hadoop-home directory ( the location where you extracted and configured hadoop )
Then apply the following command

bin/hadoop dfs -put <local file location> <hdfs file location>

or

bin/hdfs dfs -put <local file location> <hdfs file location>

You can do the same with -copyFromLocal command too. Just replace -put with -copyFromLocal in above commands.

for example :
Lets say you have pg16328.txt in your Desktop directory, then the above command would be

bin/hadoop dfs -put /home/cloudera/Desktop/pg16328.txt /user/hadoop/

where /user/hadoop is a directory in hdfs
If /user/hadoop directory doesn't exists then you can create it by

bin/hadoop dfs -mkdir -f /user/hadoop

You can look at the uploaded file using webUI ( namenodeIP:50070 ) or by using command line as

bin/hadoop dfs -ls /user/hadoop/