失败：执行错误，在Hive中进行联接操作期间，从org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask​​返回代码2

Question

I am trying to run the join query in hive on the following two tables-

select b.location from user_activity_rule a inner join user_info_rule b where a.uid=b.uid and a.cancellation=true;

Query ID = username_20180530154141_0a187506-7aca-442a-8310-582d335ad78d
Total jobs = 1
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
Execution log at: /tmp/username/username_20180530154141_0a187506-7aca-442a-8310-582d335ad78d.log
2018-05-30 03:41:51     Starting to launch local task to process map join;      maximum memory = 2058354688
Execution failed with exit status: 2
Obtaining error information

Task failed!
Task ID:
  Stage-4

Logs:

/tmp/username/hive.log
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask

What does this error mean and how to resolve this?

Answer 1

This happens when the job you are trying to run runs out of memory. one way of overcoming this is to use this command:

set hive.auto.convert.join = false;

This will help in join optimization.

Sometimes when the number of concurrent users using it is high(at some peak time), this happens. Alternatively, you can fire this query when not many users are using it. Apparently, there would be much free memory so that your job can consume required. This alternative can be adopted when nodes are less in Dev environment and you are sure that there will be no memory issues in production.

Answer 2

除了在哪里可以使用以下代码，然后尝试

SELECT b.location FROM user_activity_rule a JOIN user_info_rule b ON(a.uid=b.uid) WHERE a.cancellation="true";

Answer 3

First of all, make sure the HADOOP_USER that you used to run SQL can run MapReduce .

Then, use SQL like following:

set hive.auto.convert.join = false;
select b.location 
from user_activity_rule a 
inner join user_info_rule b 
where a.uid=b.uid and a.cancellation=true;