在 Zeppelin 中从 spark 执行代码 scala

Question

I would like to run a scala code on Zeppelin from Spark cluster.

For example: This is code into hdfs Spark "HelloWorldScala.scala":

object HelloWorldScala{
 def main (arg: Array[String]): Unit = {
 val conf = new SparkConf().setAppName("myApp_Enrico")

 val spark = SparkSession.builder.config(conf).getOrCreate()
 val aList = List(1,2,3,4,5,6,7,8,9,10)
 val aRdd = spark.sparkContext.parallelize(aList)
 println("********* HELLO WORLD AND HELLO SPARK!! ******")
 println("Print even numbers")
 aRdd.filter(x=>x%2==0).map(x=>x*2).collect().foreach(println)
  }

}

I would like to import in Zeppelin the HelloWorldScala file and run main, but I see the error:

Error code Zeppelin

Answer 1

Unfortunately you can't import single file in Zeppelin. You can pack your scala files into .jar library and put it to spark.jars (setted as property in spark) directory, after you will can import your library using line: import your.libray.packages.YourClass and using non-private functions from it. If you don't know about jar packages, and spark.jar directories just read a bit more about that.

UPDATE:

%dep
z.load("your_package_group:artifact:version")
%spark

import com.yourpackage.HelloWorldScala