在Spark Submit中访问Spark集群模式

Question

I am trying to run my spark scala code using spark submit. I want to access the spark cluster for this purpose. So, what should I use for the master in the Spark Context? I have used like this

val spark = SparkSession.builder()
      .master("spark://prod5:7077")
      .appName("MyApp")
      .getOrCreate;

But it doesn't seem to work. What should I use as the master for using spark cluster?

Answer 1

如果要在IDE中提交作业，只需确保“ prod5”是主服务器，然后尝试将端口默认更改为6066。

Answer 2

从官方文档-

spark-submit --class <main-class>  --master <master-url> --deploy-mode <deploy-mode>  --conf <key>=<value> ... # other options <application-jar>  [application-arguments]