[英]Spark Pi Example in Cluster mode with Yarn: Association lost
我有三個虛擬機作為分布式Spark集群運行。 我使用Spark 1.3.0和底層的Hadoop 2.6.0。
如果我運行Spark Pi示例
/usr/local/spark130/bin/spark-submit
--class org.apache.spark.examples.SparkPi
--master yarn-client /usr/local/spark130/examples/target/spark-examples_2.10-1.3.0.jar 10000
我收到此警告/錯誤並最終出現異常:
WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/04/08 12:37:06 WARN ReliableDeliverySupervisor: Association with remote system [akka.tcp://sparkYarnAM@virtm4:47128] has failed, address is now gated for [5000] ms. Reason is: [Disassociated].
15/04/08 12:37:12 WARN ReliableDeliverySupervisor: Association with remote system [akka.tcp://sparkYarnAM@virtm4:45975] has failed, address is now gated for [5000] ms. Reason is: [Disassociated].
15/04/08 12:37:13 ERROR YarnClientSchedulerBackend: Yarn application has already exited with state FINISHED!
當我檢查容器的日志時,我發現它是SIGTERM-ed
15/04/08 12:37:08 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:1408, vCores:1>)
15/04/08 12:37:08 INFO yarn.YarnAllocator: Container request (host: Any, capability: <memory:1408, vCores:1>)
15/04/08 12:37:08 INFO yarn.ApplicationMaster: Started progress reporter thread - sleep time : 5000
15/04/08 12:37:12 ERROR yarn.ApplicationMaster: RECEIVED SIGNAL 15: SIGTERM
15/04/08 12:37:12 INFO yarn.ApplicationMaster: Final app status: UNDEFINED, exitCode: 0, (reason: Shutdown hook called before final status was reported.)
15/04/08 12:37:12 INFO yarn.ApplicationMaster: Unregistering ApplicationMaster with UNDEFINED (diag message: Shutdown hook called before final status was reported.)
解決方案:我解決了這個問題。 我現在使用Java7而不是Java8。 這種情況被報告為bug,但它被拒絕了這樣的https://issues.apache.org/jira/browse/SPARK-6388然而,改變Java版本確實有效。
由於Java 8過多的內存分配問題,關聯可能會丟失: https : //issues.apache.org/jira/browse/YARN-4714
您可以通過在yarn-site.xml中設置以下屬性來強制YARN忽略它
<property>
<name>yarn.nodemanager.pmem-check-enabled</name>
<value>false</value>
</property>
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
</property>
之前我遇到過類似的問題,直到找到這個問題
嘗試顯式停止SparkContext實例sc.stop()
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.