使用 Hive 表的 Hive UDF

Question

I have developed a hive udf in java that works correctly, my function return the best match between the input and a column in a hive table, so it have this simplified pseudo-code:

class myudf  extends udf{

    evaluate(Text input){

        getNewHiveConnection(); //i want to replace this by getCurrentHiveUserConnetion();
        executeHiveQuery(input);
        return something;
}

My question is, if this function is invoked by Hive, why i need to connect to hive in my code ? can i use the current connection that the user who use my function is connected by ?

Answer 1

If you want to return a closest match from entire column in a query, you could think if it as some sort of aggregation and use Hive UDAF: https://cwiki.apache.org/confluence/display/Hive/GenericUDAFCaseStudy

There's also quite handy tutorial: http://beekeeperdata.com/posts/hadoop/2015/08/17/hive-udaf-tutorial.html

Answer 2

Another way will be to create Macros. They work both on Hive and Beeline.

CREATE TEMPORARY MACRO fn_maskNull(input decimal(25,3))
CASE
    WHEN input IS NULL THEN 0 else input
END;

-- usage
select fn_maskNull(null), fn_maskNull(101);

More info :

https://medium.com/@gchandra/create-user-defined-functions-in-hive-beeline-ff965285d735

Answer 3

Yes - you can make the UDF permanent. For example:

CREATE FUNCTION MatchFinder as 'com.mycompany.packagex.myudf' using jar  'hdfs:///an_HDFS_directory/my_jar_name.jar';

This will make your function permanent and anyone will be able to call it. In this case, the jar file is stored on HDFS for easy accessibility, but there are other options.

See Hive wiki for more details.