简体   繁体   中英

IOException while running mapreduce job

I created a mapreduce job to calculate average of some monitoring information based on certain time periods. It was working fine. Yesterday the hadoop running machine unexpectedly shutdown due to power failure. But after restarting, while running same mapreduce job, I got the following exception:



    14/01/06 13:16:39 INFO mapred.JobClient: Running job: job_201401060557_0001
    14/01/06 13:16:40 INFO mapred.JobClient:  map 0% reduce 0%
    14/01/06 13:17:00 INFO mapred.JobClient:  map 99% reduce 0%
    14/01/06 13:17:08 INFO mapred.JobClient: Task Id : attempt_201401060557_0001_m_000000_0,

          Status : FAILED
 org.apache.hadoop.ipc.RemoteException: java.io.IOException: failed to create file  
    /user/root/out1/_temporary/_attempt_201401060557_0001_m_000000_0/hi/101-m-00000 on  
    client 192.168.1.149 either because the filename is invalid or the file exists
    at 
 org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1265)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1188)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:628)
    at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at     org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)

where /usr/root/out1 is the output path set for this job, and it was cleared before running job. Also I had shutdown this machine many times, but i didn't get such exception. I am afried my data have lost! Also it is a single node cluster.

Hoping ur help

Sorry,it was my mistake. I set the wrong combiner!.When I corrected it , it worked.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM