Remove useless dependencies in Hive UDF

Question

I'm writing a set of User Defined Functions (UDF) for Hive in a Java Maven project.

I'm using maven assembly plugin to generate an uber-jar. However, I'm also including tons of dependencies I'm sure hive already has on it's classpath, SLF4j for instance.

Is there a simple way (tool) to know what's already on the hive classpath ? I'm using HDP 2.2 from Horton Works

Answer 1

dependency:analyze in Maven Dependency Plugin will help.

dependency:analyze analyzes the dependencies of this project and determines which are: used and declared; used and undeclared; unused and declared.

dependency:analyze

dependency:tree is another thing that might help. This has tools to allow you to do the analysis yourself.

dependency:tree displays the dependency tree for this project.

dependency:tree

Answer 2

The Query to get a list of all jars that are included in the ClassPath of Hive is:

hive> list jars;

The output will look like

file:/usr/local/…..*.jar
/usr/local/…..*.jar

The jar location with file: prefix is the one which was available in the ClassPath by default. The others were added explicitly by users using the add jar command.