[英]Is there any existing API to read and write to HDFS from JAVA
是否存在可從HDFS讀取/寫入的任何現有API,以及實現該API的最佳做法。
下面是代碼片段
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
import org.apache.hadoop.util.Progressable;
String Dest = "/user/pkumar/test.xml";
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(URI.create(Dest),conf);
Path path = new Path(Dest);
if(!fs.exists(path)){
OutputStream out = fs.create(path, new Progressable(){
public void progress(){
System.out.print(".");
}
});
System.out.println();
IOUtils.copyBytes(connect, out, 4096, true);
}
在pom.xml中使用以下依賴項
<dependencies>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.3.0-cdh5.1.3</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>2.3.0-cdh5.1.3</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.3.2</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.2</version>
</dependency>
</dependencies>
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.