集群上的 Spark 进程是一个 java 进程

Question

I don't know much about java. Can anybody explain this line for me.

"Any, any apache Spark process that would ever work on your cluster or local machine is a JVM process. "

Answer 1

Spark is based on concept of driver and executor, where driver is sort of master and executor are slaves. There are two type of operations in spark: Transformation and action. Although only Driver can initiate one operations Executors are one who actually perform the operation. So you can say whatever happens in spark execution, driver and executor are only one responsible. And the driver and executor are nothing else but JVM process or JVM container (yarn). So every operations is executed by JVM process only ( Driver and Executor). Hope this explains your query.