Oozie示例在运行猪作业时卡住

Question

I start test oozie job on cluster with yarn "oozie job -config /usr/lib/oozie/oozie-4.0.1/examples/apps/pig/job.properties -run" and my job stuck on 0% and then send only heart beat

2014-06-25 12:51:57,800 [JobControl] INFO  org.apache.hadoop.yarn.client.RMProxy  - Connecting to ResourceManager at localhost/127.0.0.1:8032
2014-06-25 12:51:57,831 [JobControl] INFO  org.apache.hadoop.conf.Configuration.deprecation  - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-06-25 12:51:58,021 [JobControl] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat  - Total input paths to process : 1
2014-06-25 12:51:58,021 [JobControl] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil  - Total input paths to process : 1
2014-06-25 12:51:58,021 [JobControl] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil  - Total input paths to process : 1
2014-06-25 12:51:58,022 [JobControl] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil  - Total input paths (combined) to process : 1
2014-06-25 12:51:58,022 [JobControl] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil  - Total input paths (combined) to process : 1
2014-06-25 12:51:58,034 [JobControl] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - number of splits:1
2014-06-25 12:51:58,084 [JobControl] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - Submitting tokens for job: job_1403700612904_0002
2014-06-25 12:51:58,084 [JobControl] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - Kind: mapreduce.job, Service: job_1403700612904_0001, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@63ad6884)
2014-06-25 12:51:58,085 [JobControl] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - Kind: RM_DELEGATION_TOKEN, Service: 127.0.0.1:8032, Ident: (owner=test, renewer=oozie mr token, realUser=root, issueDate=1403700700415, maxDate=14043$
2014-06-25 12:51:58,352 [JobControl] INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl  - Submitted application application_1403700612904_0002 to ResourceManager at localhost/127.0.0.1:8032
2014-06-25 12:51:58,393 [JobControl] INFO  org.apache.hadoop.mapreduce.Job  - The url to track the job: http://localhost:8088/proxy/application_1403700612904_0002/
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - HadoopJobId: job_1403700612904_0002
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - HadoopJobId: job_1403700612904_0002
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - Processing aliases A,B
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - Processing aliases A,B
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - detailed locations: M: A[18,4],B[19,4] C:  R:
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - detailed locations: M: A[18,4],B[19,4] C:  R:
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_1403700612904_0002
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_1403700612904_0002
2014-06-25 12:51:58,429 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - 0% complete
2014-06-25 12:51:58,429 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - 0% complete
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat

Answer 1

Me too faced same problem while scheduling pig-0.12.1 in oozie-4.0.1 with hadoop-2.2.0.

I cant able to schedule pig script using oozie in hadoop-2.2.0 in single node cluster. But I did it in multinode cluster by doing the following changes.

NodeManager and Resource Manager running in same system. So i am gettting this error. I started Node manager in second system and my problem is solved.

Answer 2

Hi Heart beat error is occurs due to insufficient Total memory size in Hadoop . This because you may run in small cluster machine.

Solution: So you want to increase the size of Total memory to all nodemanager to perform mapreduce job. Steps are followed in this link "HEART BEAT ERROR SOLUTION"