[英]Check values of org.apache.spark.sql.Row
我有以下作为collect
的结果
Array[org.apache.spark.sql.Row] = Array([16660,23517,23518,10,1,0])
我如何从这个数组中提取信息? 我需要检查是否有任何值是0
val df = Seq((1,2,3,4,5),(1,0,3,4,5),(1,2,3,4,0)).toDF()
df.show()
+---+---+---+---+---+
| _1| _2| _3| _4| _5|
+---+---+---+---+---+
| 1| 2| 3| 4| 5|
| 1| 0| 3| 4| 5|
| 1| 2| 3| 4| 0|
+---+---+---+---+---+
val arr = df.collect()
// array with boolens indicating if 0 exist in each column
val res = Array.fill(arr(0).length)(false)
// convert Array[Row] to Array[Array[Int]]
val arr2 = for( j <- 0 until arr.length) yield {for (i <- 0 until arr(j).length) yield {(arr(j).getInt(i))}}
arr2.foreach(s=>for(i <- 0 until s.length) if(s(i)==0) res(i)=true)
output:
res: Array[Boolean] = Array(false, true, false, false, true)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.