How to read a serialized object from a file in HDFS in Hadoop

Question

I have serialized an object of type LibSVM into a file named j48.model. This file has been transferred into the HDFS file system.

Now, in a hadoop mapreduce code, how can I deserialize this object and read it back into an instance of class LibSVM? I have included the .jar file relevant to LIBSVM already into the ma reduce project as an external jar file.

What JAVA methods help me read the contents of the file j48.model into a LibSVM object?

Answer 1

When you open a HDFS URL, it returns you an InputStream.

FileSystem fs = FileSystem.get(new Configuration());
InputStream in = fs.open(new Path("your uri"));

You can wrap this input stream in an ObjectInputStream:

ObjectInputStream objReader = new ObjectInputStream(in);

and read your object from it:

LibSVM lib = (LibSVM)objReader.readObject();