如何在执行程序节点中以编程方式找到Spark版本？

Question

I'm trying to write a method (that will be ran through an executor) that will return the Spark version as a string. I know that I can find the Spark version with the following code:

SparkSession.builder().getOrCreate().version (even on executor)

But, when I'm running tests (tests in Apache Spark source code, were written before mine), some tests fail with the following error:

Caused by: java.lang.IllegalStateException: SparkSession should only be created and accessed on the driver.

So, I understand that I can't use SparkSession. Therefore my question is, is there any other way to find Spark version in an executor?

Answer 1

I solved my problem by importing SPARK_VERSION directly:

import org.apache.spark.SPARK_VERSION

There was also an option to transfer version with configuration that was already in my class.