How can I pretty print a wrappedarray in Zeppelin/Spark/Scala?

Question

In this question I was told how to print a dataframe using zeppelin's z.show command. This works well except for 'WrappedArray' appearing in the lemma column:

I have tried this:

z.show(dfLemma.select(concat_ws(",", $"lemma")))

but it just gave me a list of words, not nicely formatted and I also want the racist column in my output. Any help is much appreciated.

Answer 1

Here's a suggestion for formatting your array column:

import org.apache.spark.sql.Column
import org.apache.spark.sql.functions._
import sqlContext.implicits._

val df = Seq(
  (1, Array("An", "Array")), (2, Array("Another", "Array"))
).toDF("first", "second")

def formatArrayColumn(arrayColumn: Column): Column = {
  concat(lit("["), concat_ws(", ", arrayColumn), lit("]")).as(s"format(${arrayColumn.expr})")
}

val result = df.withColumn("second", formatArrayColumn($"second"))

z.show(result)

Which results in: