Scala compiler says “No TypeTag available for T” in method using generics

Question

The following code is not compiling:

override def read[T <: Product](collection : String): Dataset[T] = {
  val mongoDbRdd = MongoSpark.load(sparkSession.sparkContext,MongoDBConfiguration.toReadConfig(collection))
  mongoDbRdd.toDS[T]
}

This is "toDS" definition:

def toDS[T <: Product: TypeTag: NotNothing](): Dataset[T] = mongoSpark.toDS[T]()

Compiler says:

Error:(11, 20) No TypeTag available for T
    mongoDbRdd.toDS[T]

Error:(11, 20) not enough arguments for method toDS: (implicit evidence$3: reflect.runtime.universe.TypeTag[T], implicit evidence$4: com.mongodb.spark.NotNothing[T])org.apache.spark.sql.Dataset[T].
Unspecified value parameters evidence$3, evidence$4.
    mongoDbRdd.toDS[T]

Line 11 is mongoDbRdd.toDS[T]

I really don't know what's going on with Scala Generics and the compiler is not very specific, any idea?

Answer 1

The problem is with the type constraints on T that toDS requires:

// The ':' constraint is a type class constraint.
def toDS[T <: Product: TypeTag: NotNothing](): Dataset[T] =
  mongoSpark.toDS[T]()

// The below is exactly the same as the above, although with user-defined
// names for the implicit parameters.
// All a type class does is append implicit parameters to your function.
def toDS[T <: Product]()(implicit typeTag: TypeTag[T], notNothing: NotNothing[T]) =
  mongoSpark.toDS[T]()

You'll notice that's what your compiler error shows - with the names expanded to evidence$3 and evidence$4 .

If you want your method to compile, simply add the same type classes:

override def read[T <: Product: TypeTag: NotNothing](
    collection : String): Dataset[T] = { /* impl */ }