在使用RDD的Scala中，如果RDD [（k，Iterable [v]），如何在Iterable中獲得Apply函數

Question

我試圖為給定的RDD [Key，Iterable [Value]]類有效地在Iterable [Value]上應用過濾器（多個）的方法。

原因是我想過濾RDD並最終找到與過濾器匹配的鍵

RDD示例

 000473643-02,CompactBuffer((glucose,80.0), (glucose,80.0), (glucose2,80.0),   (fasting blood glucose,80.0), (glucose,80.0), (glucose,80.0), (glucose,80.0), (glucose,80.0)))
 (713003448-01,CompactBuffer((glucose,80.0), (glucose,80.0)))
 (000023838-01,CompactBuffer((glucose,80.0), (glucose,80.0)))
 (000772974-01,CompactBuffer((glucose,80.0), (glucose,80.0), (glucose,80.0)))
 (380670000-01,CompactBuffer((glucose,80.0), (glucose,80.0)))

因此，在這種情況下，僅在滿足以下條件時才需要輸出Key：

    glucose value is >= 80 or fasting blood glucose >= 80 

Answer 1

我會用這樣的東西：

case class ExceedsCondition(threshold:Double) {
  def violates(value:Double) = value >= threshold
}

val conditionsBroadcast = sc.broadcast(
  Map("glucose" -> ExceedsCondition(80.0), 
      "fasting-glucose" -> ExceedsCondition(81.0))
  )

val rdd = sc.parallelize(List("key1" -> List(("glucose" -> 83.0))))

val result = rdd.filter { case (_, xs) =>
            val conditions = conditionsBroadcast.value
            xs.exists { case (key, value) =>
                conditions.get(key).exists(_.violates(value))
            }
}

result.take(10)