[英]Not able to open HDFS file from mapper Hadoop
I searched a lot but failed to find a solution to this problem. 我进行了很多搜索,但未能找到解决该问题的方法。 Actually the file I want to access is in HDFS, but not in input path (the path which was input to the map/reduce job). 实际上,我要访问的文件位于HDFS中,但不在输入路径(输入到映射/缩小作业的路径)中。 And I want to access it from mapper. 我想从映射器访问它。 The hdfs path specified in the input path is perfectly accessible from mapper but the other hdfs files are not. 输入路径中指定的hdfs路径可以从mapper完全访问,而其他hdfs文件则不能。
INside mapper:- 内部映射器:-
FileSystem FS1=FileSystem.get(conf);
Path path=new Path(""+FS1.getHomeDirectory());
FSDataInputStream fsdis=FS1.open(path);
RESULTS IN the following ERROR: java.io.IOException : Cannot open filename /user/hadoop 结果出现以下错误:java.io.IOException:无法打开文件名/ user / hadoop
Thanks in advance, Harsh 在此先感谢,苛刻
I remember using this tutorial to get something similar working. 我记得使用本教程可以获得类似的效果。 You can give it a try, it has only a few difference tho what you've written but still it might help... 您可以尝试一下,它与您所写的内容只有几处不同,但仍可能会有所帮助...
@Edit: ah and I just noticed (after reading the comments) that you are trying to open FS1.getHomeDirectory()
and that is a directory. @Edit:啊,我刚刚(在阅读注释后)注意到您正在尝试打开FS1.getHomeDirectory()
并且这是一个目录。 You should point out to a file not a directory , I think (you can check it out in the linked tutorial under "Reading data from a file"). 我想,您应该指出一个文件而不是目录 (您可以在链接的教程中的“从文件读取数据”下进行检查)。
can u try this once 你能试一次吗
try {
FileSystem fs = FileSystem.get (new Configuration ());
FileStatus[] status = fs.listStatus (new Path ("hdfs://jp.seka.com:9000/user/jeka/in"));
for (int i=0;i < status.length;i++) {
BufferedReader br = new BufferedReader (new InputStreamReader (fs.open (status[i].getPath())));
String line;
line = br.readLine();
while (line != null) {
System.out.println (line);
line=br.readLine ();
}
}
} catch (Exception e) {
System.out.println("File not found");
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.