Scala：检查通用存在类型

Question

I have a Scala library method that I need to override:

def transform(dataset: Dataset[_]): DataFrame

What I want to do: in my overridden implementation, check if _ is MyType . If yes, cast dataset to Dataset[MyType] . If not, throw an exception saying type ActualTypeOf_ is not supported .

There's only one MyType I care about. Obviously, I can't rewrite the library to change method's signature.

Answer 1

As you pointed in a comments section to the question Dataset is actually Spark Dataset. To be honest, I'm not the Spark expert, but from the doc I see there is method which allows to get Classtag for type T (as far as I understand) : https://spark.apache.org/docs/latest/api/java/org/apache/spark/sql/Dataset.html#classTag--

So, I think your solution might look like:

override def transform(dataset: Dataset[_]): DataFrame = {
 val dataClass = dataset.classTag.runtimeClass
 if (dataClass == classOf[MyType]) {
   val myTypeDataset = dataset.as[MyType]
   // further handling of `myTypeDataset`
 } else {
   throw new RuntimeException(s"type $dataClass is not supported")
  }
}

Hope this helps!