Hive on Spark CDH5.7执行错误

Question

I've updated my cluster to CDH 5.7 recently and I am trying to run a Hive query processing on Spark.

I have configured the Hive client to use the Spark execution engine and Hive Dependency on a Spark Service from Cloudera Manager.

Via HUE, i'm simply running a simple select query but seem to get this error at all times: Error while processing statement: FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.spark.SparkTask

Following are the logs for the same:

ERROR operation.Operation: Error running hive query: 
org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.spark.SparkTask
    at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:374)
    at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:180)
    at org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:72)
    at org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:232)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
    at org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:245)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

Any help to solve this would be great!

Answer 1

This problem is due to a open JIRA: https://issues.apache.org/jira/browse/HIVE-11519 . You should use another serialization tool..

Answer 2

Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.spark.SparkTask

is not the real error message, you'd better turn on the DEBUG info by using hive cli, like

bin/hive --hiveconf hive.root.logger=DEBUG,console

and you will get more detailed logs, such as, those are something i got before:

16/03/17 13:55:43 [fxxxxxxxxxxxxxxxx4 main]: INFO exec.SerializationUtilities: Serializing MapWork using kryo
java.lang.NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer$.handledType()Ljava/lang/Class;

this is caused by some dependency conflicts, see https://issues.apache.org/jira/browse/HIVE-13301 for detail.