如何使用gedit打开HDFS输出文件？

Question

I have installed and executed an mapreduce program successfully in my system(Ubuntu 14.04).

I can see the output file as,

hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -ls /user/hadoopuser/MapReduceSample-output
Found 3 items
-rw-r--r--   1 hadoopuser supergroup          0 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/_SUCCESS
drwxr-xr-x   - hadoopuser supergroup          0 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/_logs
-rw-r--r--   1 hadoopuser supergroup     880838 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/part-00000

And I can open it on terminal using following command,

hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -cat /user/hadoopuser/MapReduceSample-output/part-00000

I can see the output file on terminal, but I can't see the full result because my output has large amount of lines.

So I want to open it on gedit or nano .

Need Solution.

Answer 1

您还可以使用getmerge将HDFS文件复制到本地系统。

hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -getmerge /user/hadoopuser/MapReduceSample-output/part-00000 /home/arul/MROutput

hadoop dfs -getmerge /path/to/HDFS /path/to/save

Answer 2

instead of looking for plugin. You can add jar files from $HADOOP_INSTALL/bin in eclipse and compiler issues must be gone.

Answer 3

You can't access HDFS file from local machine(system user), so that you can't open HDFS file using gedit .

To open in gedit you have to copy to local machine.

To do that, open terminal( Ctrl + Alt + T ) and use copyToLocal a Hadoop Shell Command to copy the output file into local machine.

Do the following,

hadoopuser@arul-PC:/usr/local/hadoop$ sudo bin/hadoop dfs -copyToLocal /user/hadoopuser/MapReduceSample-output/part-00000 /home/arul/Downloads/

Now you can open the output file using gedit as follows,

$ sudo gedit /home/arul/Downloads/part-00000

Note :

1. My HDFS username is hadoopuser .
2. You can move a file from HDFS to local machine. The Hadoop Shell Command fs -mv allow to move different HDFS location.
3. For more Hadoop Shell Commands(click here) .

Update ( An another option to do the same from Y-Prithvi 's post )

you can also use getmerge to copy HDFS file to local system.

hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -getmerge /user/hadoopuser/MapReduceSample-output/part-00000 /home/arul/MROutput

hadoop dfs -getmerge /path/to/HDFS /path/to/save

Answer 4

Eclipse Setup for Hadoop Development

this should help