Java项目如何连接到本地Hadoop实例

Question

I am pretty new to hadoop and am trying to create a java project that uses hadoop.

I have hadoop running in single node cluster and I have a java project where I have imported hadoop as external jar in the build path.

Does the following make sense: How should i connect the java project to the local instance of hadoop?

Thank you

Answer 1

That should work. You don't have to do much in order to connect to your local Hadoop setup. Just create a Configuration object and tell your code where to look for your configuration files using Configuration.addResource() . A small example :

public class CopyToHdfs {

    public static void main(String[] args) throws IOException {

        Configuration conf = new Configuration();
        conf.addResource(new Path("/Users/miqbal1/hadoop-eco/hadoop-1.1.2/conf/core-site.xml"));
        conf.addResource(new Path("/Users/miqbal1/hadoop-eco/hadoop-1.1.2/conf/hdfs-site.xml"));
        FileSystem fs = FileSystem.get(conf);
        fs.copyFromLocalFile(new Path("file:///Users/miqbal1/input.txt"), new Path("/"));

    }
}

Answer 2

I think what you need is the hadoop eclipse plugin. I think it doesn't work with all the hadoop versions, but I have it running with hadoop 1.2.0 version. Take a look at this tutorial .