如何在spark 2（java）中创建广播变量？

Question

In Spark 1 we can use the below code to create a Spark broadcast variable:

SparkConf conf = new SparkConf();
conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer");

JavaSparkContext sc = new JavaSparkContext("local", "JavaAPISuite", conf);

ArrayList<Strin'valuesg> sampleList = new ArrayList<String>();
sampleList.add("value");

final Broadcast<ArrayList<String> broadcastVar = sc.broadcast(sampleList);

How can we do the same in Spark 2 using the API shown below?

SparkSession sc = SparkSession.setappName("SparkApp").getorcreate();
sc.sparkcontext().broadcast(T value, scala.reflect.ClassTag<T> evidence$11)

Answer 1

For example if you want to broadcast class named Test it should implement java.io.Serializable and then you do:

import scala.reflect.ClassTag;

ClassTag<Test> classTagTest = scala.reflect.ClassTag$.MODULE$.apply(Test.class);
Broadcast<Test> broadcastTest = sc.sparkcontext().broadcast(new Test(), classTagTest);