从getOrElse返回等于的Scala映射中类型不匹配

Question

I have this Scala snippet from a custom mapper (for use in a Spark mapPartitions ) I'm writing to compute histograms of multiple Int fields simultaneously.

def multiFeatureHistogramFunc(iter: Iterator[Row]) : Iterator[(Int, (Int, Long))] = {
    var featureHistMap:Map[Int, (Int, Long)] = Map()
    while (iter.hasNext)
    {
        val cur = iter.next;

        indices.foreach( { index:Int => 
            val v:Int = if ( cur.isNullAt(index) ) -100 else cur.getInt(index)
            var featureHist:Map[Int, Long] = featureHistMap.getOrElse(index, Map())
            val newCount = featureHist.getOrElse(v,0L) + 1L
            featureHist += (v -> newCount)
            featureHistMap += (index -> featureHist)
        })
    }
    featureHistMap.iterator
}

But the error I'm getting is this:

<console>:49: error: type mismatch;
 found   : Equals
 required: Map[Int,Long]
               var featureHist:Map[Int, Long] = 
 featureHistMap.getOrElse(index, Map())
                         ^

I couldn't find the answer to this specific issue. It looks to me like the default parameter in featureHistMap.getOrElse is a different type than the value field of the featureHistMap itself and the common parent type is Equals so this causes a type mismatch. I tried a number of different things like changing the default parameter to be a more specific type, but this just caused a different error.

Can someone explain what's going on here and how to fix it?

Answer 1

The problem is that you declared your featureHistMap as Map[Int, (Int, Long)] - note that you are mapping an Int to a pair (Int, Long) . Later, you try to retrieve a value from it as a Map[Int, Long] , instead of a pair (Int, Long) .

You either need to redeclare the type of featureHistMap to Map[Int, Map[Int, Long]] , or the type of featureHist to (Int, Long) .