Hive階乘UDF

Question

我正在嘗試在Hive中查找數字的階乘。 當前沒有Hive函數可以執行此操作，因此我嘗試編寫自己的函數。 這是我的代碼：

package com.guy.hive.udf;

import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.LongWritable;
import org.apache.commons.math3.util.ArithmeticUtils;


public final class Factorial extends UDF {

public LongWritable evaluate(final LongWritable s){
        int n = (int) s.get();
        int fact = (int) ArithmeticUtils.factorial(n);
        return new LongWritable(fact);
    }
}

當我運行此Hive查詢時：

select factorial(c) from (select count(*) as c from test_table) ;

我得到例外：

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute method public org.apache.hadoop.io.LongWritable com.vm.hive.udf.Factorial.evaluate(long)  on object com.vm.hive.udf.Factorial@37483748 of class com.vm.hive.udf.Factorial with arguments {39514210:java.lang.Long} of size 1

有人能幫忙嗎？

堆棧跟蹤：

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute method public org.apache.hadoop.io.LongWritable com.vm.hive.udf.Factorial.evaluate(org.apache.hadoop.io.LongWritable)  on object com.vm.hive.udf.Factorial@5faa5faa of class com.vm.hive.udf.Factorial with arguments {39514210:org.apache.hadoop.io.LongWritable} of size 1
        at org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:1030)
        at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.evaluate(GenericUDFBridge.java:181)
        at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
        at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
        at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
        at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:80)
        at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:842)
        at org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1052)
        at org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1077)
        ... 10 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:1006)
        ... 19 more
Caused by: org.apache.commons.math3.exception.MathArithmeticException: arithmetic exception
        at org.apache.commons.math3.util.ArithmeticUtils.factorial(ArithmeticUtils.java:317)
        at com.vm.hive.udf.Factorial.evaluate(Factorial.java:50)
        ... 24 more

[編輯1-將導入添加到Java代碼中。]

[編輯2-添加了StackTrace

Answer 1

我明白你的問題。 問題不在於Hive，而在於ArithmeticUtils factorial方法。 看到它拋出了MathArithmeticException嗎？ 根據文檔，當“結果太大而無法用長整數表示”時，就會出現這種情況。

這一定是您的情況。 嘗試將較小的數字傳遞給該方法。

另外，請注意不推薦使用factorial方法。 文檔建議改為使用CombinatoricsUtils.factorialLog（int）方法。