![](/img/trans.png)
[英]Spark - Convert a coulmn containing JSON string from StringType to ArrayType(StringType())
[英]Convert datatype of cloumn from StringType to StructType in dataframe in spark scala
| ID|CO_ID| DATA|
+--------------------+--------------------+----+
|ABCD123|abc12|[{"month":"Jan","day":"monday"}] |
|BCHG345|wed34|[{"month":"Jul","day":"tuessay"}]|
我上面有數據框,其中DATA是StringType.I希望它轉換為StructType。 我怎樣才能做到這一點?
df.withColumn("data_struct",from_json($"data",StructType(Array(StructField("month", StringType),StructField("day", StringType)))))
在Spark 2.4.0上,我得到以下內容
import org.apache.spark.sql.types.{StructType, StructField, StringType}
val df = List ( ("[{\"month\":\"Jan\",\"day\":\"monday\"}]")).toDF("data")
val df2 = df.withColumn("data_struct",from_json($"data",StructType(Array(StructField("month", StringType),StructField("day", StringType)))))
df2.show
+--------------------+-------------+
| data| data_struct|
+--------------------+-------------+
|[{"month":"Jan","...|[Jan, monday]|
+--------------------+-------------+
df2.printSchema
root
|-- data: string (nullable = true)
|-- data_struct: struct (nullable = true)
| |-- month: string (nullable = true)
| |-- day: string (nullable = true)
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.