Move files to HDFS using Java

Question

How can I execute hadoop put files in hdfs using Java? That's possible?

Using this statement:

public abstract boolean rename(Path src, Path dst) throws IOException

?

Thanks!

Answer 1

Try this:

//Source file in the local file system
String localSrc = args[0];
//Destination file in HDFS
String dst = args[1];

//Input stream for the file in local file system to be written to HDFS
InputStream in = new BufferedInputStream(new FileInputStream(localSrc));

//Get configuration of Hadoop system
Configuration conf = new Configuration();
System.out.println("Connecting to -- "+conf.get("fs.defaultFS"));

//Destination file in HDFS
FileSystem fs = FileSystem.get(URI.create(dst), conf);
OutputStream out = fs.create(new Path(dst));

//Copy file from local to HDFS
IOUtils.copyBytes(in, out, 4096, true);

Answer 2

You should be able to use copyFromLocalFile :

Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
Path localPath = new Path("path/to/local/file");
Path hdfsPath = new Path("/path/in/hdfs");
fs.copyFromLocalFile(localPath, hdfsPath);