[英]Exception in thread “delete Spark local dirs” java.lang.NullPointerException
嗨,我正在通過Shell腳本運行Sparkr Progrm。 我將輸入文件指向本地意味着它工作正常,但是當我指向hdfs時,它會引發錯誤。
Exception in thread "delete Spark local dirs" java.lang.NullPointerException
Exception in thread "delete Spark local dirs" java.lang.NullPointerException
at org.apache.spark.storage.DiskBlockManager.org$apache$spark$storage$DiskBlockManager$$doStop(DiskBlockManager.scala:161)
at org.apache.spark.storage.DiskBlockManager$$anon$1$$anonfun$run$1.apply$mcV$sp(DiskBlockManager.scala:141)
at org.apache.spark.storage.DiskBlockManager$$anon$1$$anonfun$run$1.apply(DiskBlockManager.scala:139)
at org.apache.spark.storage.DiskBlockManager$$anon$1$$anonfun$run$1.apply(DiskBlockManager.scala:139)
at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1617)
at org.apache.spark.storage.DiskBlockManager$$anon$1.run(DiskBlockManager.scala:139)
任何幫助將不勝感激。
我在Scala腳本中遇到了同樣的問題。 問題出在主URL,所以我刪除了設置主URL。
先前:
val conf = new org.apache.spark.SparkConf().setMaster(masterURL).set("spark.ui.port",port).setAppName("TestScalaApp")
固定代碼:
val conf = new org.apache.spark.SparkConf().setAppName("TestScalaApp")
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.