Scala error Could not find implicit value for parameter

Question

I have the function flagVectorOutlier as showed in the code bellow. I'm using Breeze's DenseVector and DenseMatrix objects to calculate the value of distance . I expect as coded on the function signature, to get a Spark RDD[(Double, Boolean)] . mi and invCovMatrix are respectively Breeze's DenseVector[Double] and DenseMatrix[Double] :

  def flagVectorOutlier(testVectors: RDD[(String, SparkVector)], distanceThreshold: Double): RDD[(Double, Boolean)] = {
    val testVectorsDenseRDD = testVectors.map { vector => DenseVector(vector._2.toArray)}
    val mahalanobisDistancesRDD = testVectorsDenseRDD.map { vector =>
      val distance = DenseVector[Double](DenseVector(Transpose(vector - mi) * invCovMatrix) * DenseVector(vector - mi)).toArray
      (distance(0), if(distance(0) >= distanceThreshold) true else false)
    }

    mahalanobisDistancesRDD
  }

The compiler ends up showing me the following 2 errors:

Error:(75, 93) could not find implicit value for parameter op: breeze.linalg.operators.OpMulMatrix.Impl2[breeze.linalg.DenseVector[breeze.linalg.Transpose[breeze.linalg.DenseVector[Double]]],breeze.linalg.DenseVector[breeze.linalg.DenseVector[Double]],That]
      val distance = DenseVector[Double](DenseVector(Transpose(vector - mi) * invCovMatrix) * DenseVector(vector - mi)).toArray
                                                                                            ^

and

Error:(75, 93) not enough arguments for method *: (implicit op: breeze.linalg.operators.OpMulMatrix.Impl2[breeze.linalg.DenseVector[breeze.linalg.Transpose[breeze.linalg.DenseVector[Double]]],breeze.linalg.DenseVector[breeze.linalg.DenseVector[Double]],That])That.
Unspecified value parameter op.
      val distance = DenseVector[Double](DenseVector(Transpose(vector - mi) * invCovMatrix) * DenseVector(vector - mi)).toArray
                                                                                            ^

What point am I missing? I'm considering that's possible to do a multiplication between Breeze's DenseVector 's in that way.

Answer 1

If you look at the error message carefully, it tells you what went wrong.

(implicit op: OpMulMatrix.Impl2[
       DenseVector[Transpose[DenseVector[Double]]],
       DenseVector[DenseVector[Double]],
       That])

The operands for your multiply are almost certainly not what you intended. The LHS is a DenseVector whose elements are Transpose'd DenseVectors. The RHS is a DenseVector[DenseVector[Double]] . Your outer DenseVector(..) calls are wrapping their arguments in new DenseVectors, rather than turning the argument into a DenseVector.

I believe this is what you wanted:

val diff = (vector - mi).toDenseVector
(diff.t * invCovMatrix * diff)

Answer 2

Breeze relies heavily on implicits (read on those if you are not familiar with them) in order to determine compatible types for different operations. Clearly, the operator you are calling requires such an implicit parameter and the compiler is not being able to find one in scope. So there are two options:

1. You are missing an import which will produce the implicit parameters the method is missing.

2. You are trying to multiply incompatible types, and the proper implicit parameter does not exist. For instance, this tutorial mentions the following:

Assignments with incompatible cardinality or a larger numeric type won't compile.

scala> m := x :13: error: could not find implicit value for parameter op: breeze.linalg.operators.BinaryUpdateOp[breeze.linalg.DenseMatrix[Int],breeze.linalg.DenseVector[Double],breeze.linalg.operators.OpSet] m := x ^

On one hand is nice to have the compiler be able to pick this kind of errors of compatibility, but sadly the error you get is hardly very telling.

Answer 3

you should declare the type explicitly as following,focus on variables a and b :

package com.tencent.ieg.dm.demo
import breeze.linalg.{Vector, DenseVector, SparseVector}

object BreezeDemo extends App {

    val a:Vector[Long] = DenseVector(2, 10, 3)
    // val a = DenseVector(2, 10, 3) // cause compiling error
    val b:Vector[Long] = new SparseVector(Array(0),Array(2),3)
    val rst = a dot b
    println(rst)

}

Answer 4

I also meet this problem and Breeze gives me the same error message. In my case, my X type is DenseVector[Double] while Y type is DenseVector[Int], after I change Y type to DenseVector[Double], this error disappears. Hope this can help you.