[英]Elastic Map Reduce Error
I am getting an error when using Elastic Map Reduce and I am not sure what it means because it is not very descriptive. 使用Elastic Map Reduce时出现错误,我不确定它的含义,因为它描述性不强。 I want to know specifically what kind of JSONDecodeError I am getting.
我想特别知道我得到的是哪种JSONDecodeError。 "12" is not descriptive.
“ 12”不是描述性的。 This is the output.
这是输出。 I am using the MRjob class
我正在使用MRjob类
writing to step-0-mapper_part-00000
writing to step-0-mapper_part-00001
Counters from step 1:
Undecodable input:
JSONDecodeError: 12
Unencodable output:
TypeError: 2
writing to /tmp/emr_irene.ec2-user.20130716.171313.288877/step-0-mapper-sorted
> sort /tmp/emr_irene.ec2-user.20130716.171313.288877/step-0-mapper_part-00000 /tmp/emr_irene.ec2- user.20130716.171313.288877/step-0-mapper_part-00001
writing to step-0-reducer_part-00000
Counters from step 1:
Undecodable input:
JSONDecodeError: 12
Unencodable output:
TypeError: 2
The number 12 in this case is the number of json encode errors you had. 在这种情况下,数字12是您遇到的json编码错误的数量。 To find out specifically why they failed the best way I know how is by reducing your input set and triangulating the issue by rerunning the job.
为了弄清楚他们为什么失败了,我知道最好的方法是减少输入集并通过重新运行作业来对问题进行三角化。
如果您正在使用Amazon AWS,则可以运行以下脚本,您将获得有关错误的更多信息:
python -m mrjob.tools.emr.fetch_logs --find-failure YOUR_JOB_ID
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.