Custom ClassLoader in java for loading jar from hdfs

Question

I want to load a jar from HDFS using custom java URL ClassLoader in my program. I could not find examples on internet. I have seen examples that load jars from local file system as mentioned in following thread:

Custom URLClassLoader

Any Suggestions or working example?

Thanks.

Answer 1

this is how you can write your custom class loader. the code is written in scala. you can convert it into java .

class HdfsClassLoaderclassLoader(classLoader: ClassLoader) extends URLClassLoader(Array.ofDim[URL](0), classLoader) {

    def addJarToClasspath(jarName: String) {
        synchronized {
            var conf = new Configuration
            val fileSystem = FileSystem.get(conf)
            val path = new Path(jarName);
            if (!fileSystem.exists(path)) {
                println("File does not exists")
            }
            val uriPath = path.toUri()
            val urlPath = uriPath.toURL()
            println(urlPath.getFile)
            addURL(urlPath)
        }
    }
}