Read a properties file from HDFS

Question

I'm trying to read a Java properties file that is on HDFS like this:

try {
    properties.load(new FileInputStream("hdfs://user/hdfs/my_props.properties"));
} catch (IOException e) {
    throw new RuntimeException("Properties file not found.");
}

But it doesn't seem to work and I get the "Properties file not found." exception. If I replace the path to a local file, it works fine and I'm able to read the file.

Is it possible to read a HDFS file using FileInputStream?

Thanks!

Answer 1

I hope you need to use hadoop jars and also need the FileSystem to read from HDFS. Something like this should be put before your code.

Path pt=new Path("hdfs://user/hdfs/my_props.properties");
FileSystem fs = FileSystem.get(new Configuration());

Refer to: FileInputStream for a generic file System

Answer 2

 val fs = FileSystem.get(new Configuration)
 val hdfsPath = new Path("hdfs://user/hdfs/my_props.properties")
 val fis = new InputStreamReader(fs.open(hdfsPath))
 prop.load(fis) // This will be your properties object