The import org.apache.hadoop cannot be resolved

Question

I have seen this asked but none of the fixes seem to be working for me. Or more likely I am doing it wrong as I am very new to this.

(Working in eclipse) I get the error The import org.apache.hadoop cannot be resolved on the following lines:

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.filecache.DistributedCache;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapred.*;

I thought I'd added the correct external jars but apparently not. The ones I have are below. I added them by going to Properties of my project, Java Build Path, Libraries, Add External Jar

commons-codec-1.9.jar commons-fileupload-1.3.1.jar commons-io-2.4-javadoc.jar commons-io-2.4-sources.jar commons-io-2.4-testsources.jar commons-io-2.4-tests.jar commons-io-2.4.jar commons-logging-1.2.jar fluent-hc-4.4.jar httpclient-4.4.jar httpclient-cache-4.4.jar httpclient-win-4.4.jar httpcore-4.4.jar httpmime-4.4.jar jna-4.1.0.jar jna-platform-4.1.0.jar

Answer 1

You seem to still be missing dependencies.

Hadoop development is not trivial, there are a lot of dependencies and it can get messy very fast. Why not use a dependency manager like Maven? You could then simply add this to your POM.xml and let it do the hard work for you:

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-common</artifactId>
    <version>2.6.0</version>
</dependency>
<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-mapreduce-client-core</artifactId>
    <version>2.6.0</version>
</dependency>
<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-hdfs</artifactId>
    <version>2.6.0</version>
</dependency>

If you are not familiar with Maven, there are tons of information available on SO and elsewhere about how to setup basic Maven projects with Eclipse!