how to integrate eclipse and hadoop 2.6

Question

I've installed hadoop 2.6 in my ubuntu 14.04 but when I want to run a code in hadoop with java API, I should export a jar file and run it with terminal. Is there any way to configure eclipse with hadoop and run java code for hadoop in eclipse directly ?

Answer 1

Integrating Hadoop-2.6.0 with eclipse

1. User “hdfs” is created where all Hadoop processes are running.
2. Hadoop is installed to the directory “/opt/hadoop“.
3. Eclipse is installed to the directory “/opt/eclipse“.

Step 1: Download the hadoop-eclipse-plugin 2.6.0 jar

Step 2: Copy the Map-Reduce plugin for eclipse in the the plugins directory of your eclipse folder – sudo cp /home/hdfs/Downloads/hadoop-eclipse-plugin-2.6.0.jar /opt/eclipse/plugins/ Restart the eclipse using the command – /opt/eclipse/eclipse -vm /usr/local/jdk1.8.0_05/bin/java -vmargs -Xmx1024m If elcipse is not coming up because of the X11 forwarding issue, try using “sux” instead of “su” while switching to the “hdfs“.

Step 3: Start the eclipse $ECLIPSE_HOME/eclipse

step 4: In Eclipse menu click, Window --> Open Perspective --> Others --> MapReduce

step 5: In bottom MapReduce icon click to Add new Hadoop location

step 6: Enter MapReduce & HDFS running port For recall, MapReduce port (9001) specified in $HADOOP_HOME/conf/mapred-site.xml For recall, HDFS port (9000) specified in $HADOOP_HOME/conf/core-site.xml Enter the Hadoop user name

step 7 : Once Hadoop location added, DFS Locations will be seen/displayed in Eclipse Project Explorer window, (Windows-->Show View-->Project Explorer)

step 8: Once Hadoop added, DFS Locations will be seen/displayed in Project Explorer window,

step 9: Right click DFS location and click to Connect

step 10 : Once connected successfully, it will display all the DFS Folder.

Step 11: You can create Directory, Upload files to HDFS location, Download files to local by right click any of the listed Directory.