[英]biginsights on cloud - /*: bad substitution
I'm trying to run a spark yarn job on BigInsights on Cloud 4.2 Basic cluster but I'm hitting the following issue: 我正在尝试在Cloud 4.2 Basic集群上的BigInsights上运行火花纱作业,但遇到以下问题:
Stack trace: ExitCodeException exitCode=1: /data/hadoop-swap/yarn/local/usercache/snowch/appcache/application_1480680664469_0038/container_1480680664469_0038_01_000004/launch_container.sh: line 24: $PWD:$PWD/__spark__.jar:/etc/hadoop/conf:/usr/iop/current/hadoop-client/*:/usr/iop/current/hadoop-client/lib/*:/usr/iop/current/hadoop-hdfs-client/*:/usr/iop/current/hadoop-hdfs-client/lib/*:/usr/iop/current/hadoop-yarn-client/*:/usr/iop/current/hadoop-yarn-client/lib/*:/usr/lib/hadoop-lzo/lib/*:/etc/hadoop/conf/:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:$PWD/mr-framework/hadoop/share/hadoop/tools/lib/*:/usr/iop/${iop.version}/hadoop/lib/hadoop-lzo-0.5.1.jar:/etc/hadoop/conf/secure:/usr/lib/hadoop-lzo/lib/*: bad substitution
Digging deeper in to the error, I see: 深入研究错误,我看到:
Error: Could not find or load main class org.apache.spark.executor.CoarseGrainedExecutorBackend
The solution was here: http://www-01.ibm.com/support/docview.wss?uid=swg21987053 解决方案在这里: http : //www-01.ibm.com/support/docview.wss?uid=swg21987053
However, I just had to set the correct iop.version: 但是,我只需要设置正确的iop.version:
--conf "spark.driver.extraJavaOptions=-Diop.version=4.2.0.0"
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.