How ApplicationMaster in Client mode in Spark-yarn works?

Question

I'm quite new at spark. Currently I'm looking Spark code to better understand how it works.

As far as I understand, In Spark Client mode, org.apache.spark.deploy.SparkSubmit class launches user's class wrapped by org.apache.spark.deploy.JavaMainApplication in this code(If my class doesn't extend SparkApplication)

val app: SparkApplication = if (classOf[SparkApplication].isAssignableFrom(mainClass)) {
  mainClass.getConstructor().newInstance().asInstanceOf[SparkApplication]
} else {
  new JavaMainApplication(mainClass) //Here my class will be ran by this
}
...
app.start(childArgs.toArray, sparkConf)

But user application doesn't have any launching ApplicationMaster logic. I don't understand how ApplicationMaster is up in client mode if my class doesn't implement org.apache.spark.deploy.SparkApplication .

One thing I found out is there is the org.apache.spark.deploy.yarn.ExecutorLauncher in spark yarn package, but I have no idea how it is ran if user doesn't extend SparkApplication.

Answer 1

I managed to figure out how it works. It is initialized in SparkContext when TaskScheduler is initialized