如何修复 Spark (Scala) 中的“java.lang.Integer cannot be cast to java.lang.Double”错误？

Question

I have a 2 column (1 int and 1 double) dataframe "fit_comparison", of predicted values and linear regression results.

I have used the following code to get regression metrics:

val rm = new RegressionMetrics(
  fit_comparison.rdd.map(x =>
  (x(0).asInstanceOf[Double], x(1).asInstanceOf[Double])))

When I try to get specific regression metrics as below, I get the "java.lang.Integer cannot be case to java.lang.Double" error.

println("MSE: " + rm.meanSquaredError)

Do I first have to convert the first column of "fit_comparison" to a double?

Any help is appreciated, thank you,

Answer 1

You can try this :

val rm = new RegressionMetrics
(fit_comparison.rdd.map(x =>
   (x(0).asInstanceOf[Int].toDouble, 
    x(1).asInstanceOf[Int].toDouble)))