Exception:java.lang.Double cannot be cast to [D

Question

I have logistic regression model and I want to calculating recall@25

val predictionAndLabels = predicted
.select("prediction", "label")
.rdd.map(x => (x(0).asInstanceOf[Array[Double]], x(1)
.asInstanceOf[Array[Double]]))
val matrix = new RankingMetrics(predictionAndLabels)
 Array(1, 25).foreach { k =>
  println(s"Recall at $k = ${matrix.RecallAt(k)}")
}

this exception appear

java.lang.Double cannot be cast to [D

How to solve it?

Answer 1

RankingMetrics requires RDD(array, array) where as you are having RDD(double,double) which means you are passing the INVALID arguments.

In General for Type conversion, map transformation used

Assuming that predictionAndLabels.rdd type is RDD(double,double)

pairArrRDD = predictionAndLabels.rdd.map(x => (Array(x._1),Array(x._2))) //this is in scala

val matrix = new RankingMetrics( pairArrRDD )