java.lang.ClassCast errors.GenericRowWithSchema 不能转换为 scala.collection.Seq

Question

How to convert the RDD of map to wrapped Array , I am getting error

Schema:

When I am trying to convert dataframe to pojo, I am getting exception as below:

java.lang.ClassCastException: 
  org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot 
  be cast to scala.collection.Seq

Code:

rdd.map(row => {
  var arrm_list: Seq[Row] = rows.getAs[AnyRef]("ArrTeber").asInstanceOf[Seq[Row]]
  //working fine here
  arrm_list.foreach(x => {
    var arrmtrb__list: Seq[Row] = rows.getAs[AnyRef]("ArrTeberPRD").asInstanceOf[Seq[Row]]

    //working fine here
    arrmtrb__list.foreach(pt => {

      var pd__list: Seq[Row] = rows.getAs[AnyRef]("Pduct").asInstanceOf[Seq[Row]] //raising error
    })
  })
})

Answer 1

The above exception is simply a class cast exception, Since struct cannot be cast to Seq of struct (Refer Schema: -- Pduct: struct (nullable = true) ). Cast Pduct to Row and then extract nested members.

df.map(row => {
  var arrm_list: Seq[Row] = row.getAs[Seq[Row]]("ArrTeber")

  arrm_list.foreach(x => {
    var arrmtrb__list: Seq[Row] = x.getAs[Seq[Row]]("ArrTeberPRD")


    arrmtrb__list.foreach(pt => {
      var pd__list: Row = pt.getAs[Row]("Pduct") //Pduct: struct (nullable = true)

    })
  })

})