简体   繁体   中英

java.io.IOException: Initialization of all the collectors failed. Error in last collector was :null

I am a newbie in MapReduce and I am trying to find a solution to a problem.I am trying to chain two map reduce jobs.The first job is getting executed but on the second job I am getting an error as follows

     INFO mapreduce.Job: Task Id : attempt_1445271708293_0055_m_000000_1, Status : FAILED
Error: java.io.IOException: Initialization of all the collectors failed.              Error in last collector was :null
    at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:414)
    at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
    at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:698)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.NullPointerException
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:1011)
    at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:402)
    ... 9 more

I had imported Text in Driver class as

import com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider.Text

instead of

import org.apache.hadoop.io.Text

because of which I was getting the error, once I rectified the mistake it started working fine.

I have got the same error when output types from map task didn't matched with input types of reduce task.

Last two arguments in mapper should have the same type as first two in reducer.

public class ByteCalculationMapper extends Mapper<LongWritable, Text, Text, IntWritable> {

public class ByteCalculationReducer extends Reducer<Text, IntWritable, Text, Text> {

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM