尝试将数据从 Data Fusion 加载到 Salesforce 时出错

Question

I`m getting this error when trying to load data from Data Fusion to Salesforce:

java.lang.RuntimeException: There was issue communicating with Salesforce
    at io.cdap.plugin.salesforce.plugin.sink.batch.SalesforceOutputFormat.getRecordWriter(SalesforceOutputFormat.java:53) ~[1599122485492-0/:na]
    at org.apache.spark.internal.io.HadoopMapReduceWriteConfigUtil.initWriter(SparkHadoopWriter.scala:350) ~[spark-core_2.11-2.3.4.jar:2.3.4]
    at org.apache.spark.internal.io.SparkHadoopWriter$.org$apache$spark$internal$io$SparkHadoopWriter$$executeTask(SparkHadoopWriter.scala:120) ~[spark-core_2.11-2.3.4.jar:2.3.4]
    at org.apache.spark.internal.io.SparkHadoopWriter$$anonfun$3.apply(SparkHadoopWriter.scala:83) ~[spark-core_2.11-2.3.4.jar:2.3.4]
    at org.apache.spark.internal.io.SparkHadoopWriter$$anonfun$3.apply(SparkHadoopWriter.scala:78) ~[spark-core_2.11-2.3.4.jar:2.3.4]
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) ~[spark-core_2.11-2.3.4.jar:2.3.4]
    at org.apache.spark.scheduler.Task.run(Task.scala:109) ~[spark-core_2.11-2.3.4.jar:2.3.4]
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345) ~[spark-core_2.11-2.3.4.jar:2.3.4]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_252]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_252]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_252]
Caused by: com.sforce.async.AsyncApiException: InvalidJob : Invalid job id: null
    at com.sforce.async.BulkConnection.parseAndThrowException(BulkConnection.java:182) ~[na:na]
    at com.sforce.async.BulkConnection.doHttpGet(BulkConnection.java:753) ~[na:na]
    at com.sforce.async.BulkConnection.getJobStatus(BulkConnection.java:769) ~[na:na]
    at com.sforce.async.BulkConnection.getJobStatus(BulkConnection.java:760) ~[na:na]
    at io.cdap.plugin.salesforce.plugin.sink.batch.SalesforceRecordWriter.<init>(SalesforceRecordWriter.java:69) ~[1599122485492-0/:na]
    at io.cdap.plugin.salesforce.plugin.sink.batch.SalesforceOutputFormat.getRecordWriter(SalesforceOutputFormat.java:51) ~[1599122485492-0/:na]
    ... 10 common frames omitted
2020-09-03 08:41:28,595 - WARN  [task-result-getter-0:o.a.s.ThrowableSerializationWrapper@192] - Task exception could not be deserialized
java.lang.ClassNotFoundException: Class not found in all delegated ClassLoaders: com.sforce.async.AsyncApiException
    at io.cdap.cdap.common.lang.CombineClassLoader.findClass(CombineClassLoader.java:96) ~[na:na]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[na:1.8.0_252]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[na:1.8.0_252]
    at io.cdap.cdap.common.lang.WeakReferenceDelegatorClassLoader.findClass(WeakReferenceDelegatorClassLoader.java:58) ~[na:na] 

What does this error mean? I have assured the input fields match with the SObject definition.

Answer 1

I looked at the stacktrace, and I think I might know what the problem is.

The property mapred.salesforce.job.id is undefined, and when the code executes, it goes to grab the value of this key, and as it is not defined, the job errors out. I think you need to set the mapred.salesforce.job.id flag as a runtime property. To do that in Data Fusion, do the following:

1. In the Pipeline Studio, navigate to the detail page of the pipeline you would like to configure.
2. Click on the drop-down next to the Run button.
3. Set your desired cluster properties, prefixing all property names with system.profile.properties. . For our case, I think the name would be system.profile.properties.mapred:mapred.salesforce.job.id and the value is a number you'd want to use as your ID