检查 org.apache.spark.sql.Row 的值

Question

I have the following as a results of a collect

Array[org.apache.spark.sql.Row] = Array([16660,23517,23518,10,1,0])

How do I pull info from this Array? I need to check if any of the values is a 0

Answer 1

val df = Seq((1,2,3,4,5),(1,0,3,4,5),(1,2,3,4,0)).toDF()
df.show()
+---+---+---+---+---+
| _1| _2| _3| _4| _5|
+---+---+---+---+---+
|  1|  2|  3|  4|  5|
|  1|  0|  3|  4|  5|
|  1|  2|  3|  4|  0|
+---+---+---+---+---+


val arr = df.collect() 
// array with boolens indicating if 0 exist in each column
val res =  Array.fill(arr(0).length)(false) 
// convert Array[Row] to Array[Array[Int]]
val arr2 = for( j <- 0 until arr.length) yield {for (i <- 0 until arr(j).length) yield {(arr(j).getInt(i))}} 
arr2.foreach(s=>for(i <- 0 until s.length) if(s(i)==0) res(i)=true)

output:

res: Array[Boolean] = Array(false, true, false, false, true)