Spark Kryo Registrator

Question

I'm using kryo as serializer in spark 2.1 I have set registration as required but I have facing a problem, I can't register this class:

org.apache.spark.sql.execution.datasources.PartitioningAwareFileIndex$SerializableFileStatus$SerializableBlockLocation[]

I have tried with classOf[Array[Class.forName("org.apache.spark.sql.execution.datasources.PartitioningAwareFileIndex$SerializableFileStatus$SerializableBlockLocation").type]] and other combinations but I don't succeed in the registration of this class.

I have tried as well creating a java class like register and registering the class as org.apache.spark.sql.execution.datasources.PartitioningAwareFileIndex$SerializableFileStatus$SerializableBlockLocation[].class;

Any clue is appreciated, Thanks.

Answer 1

I have solved my question, I've used:

kryo.register(ClassTag(Class.forName("org.apache.spark.sql.execution.datasources.PartitioningAwareFileIndex$SerializableBlockLocation")).wrap.runtimeClass)

It allows register for the class name at runtime wrapped in an array.