Sqoop導出失敗-找不到ExportBatchOutputFormat

Question

當我嘗試將配置單元分區導出到我的Oracle數據庫時，導出失敗，並且我無法弄清楚為什么...有人可以幫助我嗎？

我的Oracle表：

  CREATE TABLE "GFPROGNOSE"."HADOOP_TEST" 
   ("COLUMN1" VARCHAR2(255), 
    "COLUMN2" VARCHAR2(255), 
    "COLUMN3" VARCHAR2(255), 
    "COLUMN4" VARCHAR2(255), 
    "DS_YEAR" VARCHAR2(255), 
    "DS_MONTH" VARCHAR2(255)
   )

我的Hive表：將Oracle Table的所有列都作為String和其他列

我的獨家新聞出口命令：

 sudo -u hdfs sqoop-export --connect jdbc:oracle:thin:@192.168.145.210:1521:dmine
 --table HADOOP_TEST --username USERNAME --password xxxxxx --num-mappers 1 
--update-mode allowinsert --skip-dist-cache --hcatalog-database default 
--hcatalog-table adobe_archive --hcatalog-partition-keys ds_year,ds_month,ds_day
 --hcatalog-partition-values 2017,06,01

命令行輸出：

17/08/10 10:03:56 INFO mapreduce.Job:  map 0% reduce 0%
17/08/10 10:03:56 INFO mapreduce.Job: Job job_1501840909803_1851 failed with state FAILED due to: Application application_1501840909803_1851 failed 2 times due to AM Container for appattempt_1501840909803_1851_000002 exited with  exitCode: 1
For more detailed output, check the application tracking page: http://master.joinplus.local:8088/cluster/app/application_1501840909803_1851 Then click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_e17_1501840909803_1851_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:944)
        at org.apache.hadoop.util.Shell.run(Shell.java:848)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1142)
        at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:237)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:317)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:83)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

應用日志：

2017-08-10 10:03:55,839 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
2017-08-10 10:03:55,880 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.cloudera.sqoop.mapreduce.ExportBatchOutputFormat not found
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.cloudera.sqoop.mapreduce.ExportBatchOutputFormat not found
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:520)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:500)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.callWithJobClassLoader(MRAppMaster.java:1601)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:500)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.java:286)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1559)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1556)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1487)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.cloudera.sqoop.mapreduce.ExportBatchOutputFormat not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2241)
        at org.apache.hadoop.mapreduce.task.JobContextImpl.getOutputFormatClass(JobContextImpl.java:222)
        at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:516)
        ... 11 more
Caused by: java.lang.ClassNotFoundException: Class com.cloudera.sqoop.mapreduce.ExportBatchOutputFormat not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2147)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2239)
        ... 13 more

Answer 1

現在可以了。 問題是sqoop命令中的--skip-dist-cache參數。
在那里是因為我復制了falcon為調試目的而生成的oozie工作流程中的命令。