简体   繁体   English

使用Java从hadoop读取文件:无输出

[英]reading a file from hadoop using java :no output

I am trying to read and write the file from HDFS. 我正在尝试从HDFS读取和写入文件。 First ,I am trying to read it and display it on console. 首先,我正在尝试阅读并将其显示在控制台上。 It runs without giving any error, warning but also it is not printing the data from the file on console, My file is already saved in hdfs , I can see it on user interface. 它运行时没有给出任何错误,警告,但也没有从控制台上的文件中打印数据,我的文件已经保存在hdfs中,我可以在用户界面上看到它。 I think there is some problem with the path I have given, I am not sure as I am connecting hadoop first time to java .I am using eclipse as IDE. 我认为我给的路径存在一些问题,我不确定是第一次将hadoop连接到java。我正在使用eclipse作为IDE。 Can somebody look into this? 有人可以调查一下吗?

I also tried giving 50075 port number for the name node but this also did not work. 我也尝试为名称节点提供50075端口号,但这也没有用。

Edit: Can anybody tell me how to give right path in java for accessing a file stored in hdfs? 编辑: 有人可以告诉我如何在java中提供正确的路径来访问存储在hdfs中的文件吗?

I am getting this Exception 我收到此异常

File file:/localhost:54310/user/hduser/project11/a.txt does not exist. 文件文件:/ localhost:54310 / user / hduser / project11 / a.txt不存在。

java.io.FileNotFoundException: File file:/localhost:54310/user/hduser/project11/a.txt does not exist. java.io.FileNotFoundException:文件文件:/ localhost:54310 / user / hduser / project11 / a.txt不存在。

at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:397)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
 at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427)
at newclassentry.main(newclassentry.java:16)

There is a problem related to path but I do not know what will be the right path. 存在与路径相关的问题,但我不知道正确的路径是什么。

On the localhost namenode I am accessing it in /user/hduser/project11 folder. 在本地主机名节点上,我正在/ user / hduser / project11文件夹中访问它。 The file is not empty. 该文件不为空。 Code: 码:

import java.io.*;
import java.util.*;
import java.net.*;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.conf.*;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapred.*;
import org.apache.hadoop.util.*;

public class newclassentry {

    public static void main(String [] args) throws Exception {
        try {
            Path pt = new Path("hdfs://localhost:54310/user/hduser/project11/a.txt");
            FileSystem fs = FileSystem.get(new Configuration());
            conf.addResource(new Path("/usr/local/hadoop/conf/core-site.xml"));
            BufferedReader br = new BufferedReader(new InputStreamReader(fs.open(pt)));

            String line;
            line = br.readLine();
            while ((line = br.readLine()) != null) {
                System.out.println(line);
                line = br.readLine();
            }
            br.close();
        }
        catch (Exception e) {
            System.out.println(e.getMessage());
            e.printStackTrace();
        }
    }
}

Solution: 解:

We need to add core-site.xml for accessing through hdfs conf.addResource(new Path(Vars.HADOOP_HOME +"/conf/core-site.xml")); 我们需要添加core-site.xml以便通过hdfs进行访问conf.addResource(new Path(Vars.HADOOP_HOME +“ / conf / core-site.xml”));

Thanks everybody for locating the initial problem. 谢谢大家找到最初的问题。

As there is no display on the console and you are sure file is not empty, it means there is error in opening the file which is due to the invalid path of the file. 由于控制台上没有显示,并且您确定文件不为空,这意味着由于文件路径无效而导致打开文件时出错。

You need to replace the empty catch block : 您需要更换空的catch块:

catch(Exception e){
                }

with allowing to print the stack trace so you can see the error occuring. 允许打印堆栈跟踪,以便您可以看到发生的错误。

catch(Exception e){
         e.printStackTrace();
                }

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM