[英]Hadoop Java - copy file from windows share folder server to HDFS
I want upload multiple file from Windows share folder server (eg //server_name/folder/) to my HDFS using Java 我想使用Java从Windows共享文件夹服务器(例如// server_name / folder /)上载多个文件到我的HDFS
list of methods I have tried 我尝试过的方法列表
org.apache.hadoop.fs.FileUtil set input path = //server_name/folder/ it says java.io.FileNotFoundException: File //server_name/folder/ does not exist org.apache.hadoop.fs.FileUtil设置输入路径= // server_name / folder /它说java.io.FileNotFoundException:文件// server_name / folder /不存在
FileSystem.copyFromLocalFile (i think this is from local hadoop server to hdfs server) FileSystem.copyFromLocalFile(我认为这是从本地hadoop服务器到hdfs服务器)
IOUtils.copyBytes same as fileUtil >> file does not exist 与fileUtil >>文件相同的IOUtils.copyBytes不存在
a simple File.renameTo same as fileUtil >> file does not exist 与fileUtil >>文件相同的简单File.renameTo不存在
String source_path = "\\\\server_name\\folder\\xxx.txt"; 字符串source_path =“ \\\\服务器名称\\文件夹\\ xxx.txt”;
String hdfs_path = "hdfs://HADOOP_SERVER_NAME:Port/myfile/xxx.txt"; 字符串hdfs_path =“ hdfs:// HADOOP_SERVER_NAME:Port / myfile / xxx.txt”;
File srcFile = new File(source_path); 文件srcFile = new File(source_path);
File dstFile = new File(hdfs_path); 文件dstFile =新文件(hdfs_path);
srcFile.renameTo(dstFile); srcFile.renameTo(dstFile);
Do I need to create FTP or How about using FTPFileSystem
? 我是否需要创建FTP或如何使用FTPFileSystem
?
Or anyone have better solution Or Sample Code 或任何人有更好的解决方案或示例代码
thank you 谢谢
FileSystem has copyFromLocal method: FileSystem具有copyFromLocal方法:
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
Configuration configuration = new Configuration();
configuration.set("fs.defaultFS", "hdfs://abc:9000");
FileSystem fs= FileSystem.get(configuration);
fs.copyFromLocalFile(new Path("/source/directory/"),
new Path("/user/hadoop/dir"));
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.