Hive GUDF custom unzip function test case failing with error “B cannot be cast to org.apache.hadoop.io.BytesWritable”

Question

I'm writing test case for Generic UDF custom unzip evaluate function which does unzipping of zip files. This jar is used in Hivequery. Here code for test case,

public void testEvaluate() throws HiveException, IOException {
    Unzip unzip = new Unzip();
    File resourcesDirectory = new File("src/test/resources/test.zip");
    byte[] bytes = Files.readAllBytes( resourcesDirectory.toPath() );

    ObjectInspector binaryOI = PrimitiveObjectInspectorFactory.writableBinaryObjectInspector;
    ObjectInspector[] arguments = {binaryOI};
    unzip.initialize(arguments);

    GenericUDF.DeferredObject valueObj0 = new GenericUDF.DeferredJavaObject(bytes);
    GenericUDF.DeferredObject[] args = { valueObj0 };

    unzip.evaluate(args  );}

I'm getting error as below ,

java.lang.ClassCastException: [B cannot be cast to org.apache.hadoop.io.BytesWritable

at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableBinaryObjectInspector.getPrimitiveJavaObject(WritableBinaryObjectInspector.java:49)
at Unzip.evaluate(Unzip.java:32)
at UnzipTest.testEvaluate(UnzipTest.java:96)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

The error is occuring at line when reading bytes from DeferredObject[] args,

 -  byte[] input = elementOI.getPrimitiveJavaObject( arg[0].get() );

PS: test.zip contains a text file(with test string) zipped to test.zip

Answer 1

You'll need to wrap byte[] with a Writable that Hive can work with, BytesWritable in your case.
As you can see WritableBinaryObjectInspector.getPrimitiveJavaObject expects BytesWritable object as an input, not an array of bytes.

Try instead of

GenericUDF.DeferredObject valueObj0 = new GenericUDF.DeferredJavaObject(bytes);

do the following:

GenericUDF.DeferredObject valueObj0 = new GenericUDF.DeferredJavaObject(new BytesWritable(bytes));

Reproducing your case locally I was able to retrieve byte[] inside UDF evaluate method successfully.