如何使用java动态展平spark数据帧中复杂的嵌套json

Question

我的输入 json 数据帧文件如下所示：

company: array (nullable = true)
|    |-- element: struct (containsNull = true)
|    |    |-- address: struct (nullable = true)
|    |    |    |-- city: string (nullable = true)
|    |    |    |-- county: string (nullable = true)
|    |    |    |-- latitude: string (nullable = true)
|    |    |    |-- line1: string (nullable = true)
|    |    |    |-- line2: string (nullable = true)
|    |    |    |-- longitude: string (nullable = true)
|    |    |    |-- postalCode: long (nullable = true)
|    |    |    |-- state: struct (nullable = true)
|    |    |    |    |-- code: int (nullable = true)
|    |    |    |    |-- name: string (nullable = true)
|    |    |    |-- stateOtherDescription: string (nullable = true)
|    |    |-- addressSourceOther: string (nullable = true)
|    |    |-- addressSourceType: struct (nullable = true)
|    |    |     |-- code: int (nullable = true)
|    |    |     |-- name: string (nullable = true)
|    |    |-- reasons: array (nullable = true)
|    |    |     |-- element: struct (containsNull = true)
|    |    |     |-- improve: string (nullable = true)
|    |    |     |-- far: string (nullable = true)
|    |    |     |-- home: string (nullable = true)

我想使用 spark java 动态展平它。 有人可以帮我弄这个吗

Answer 1

查看http://stefanfrings.de/bfUtilities/bfUtilities.zip中的 de.stefanfrings/parsing/JsonSlurper 类。 它读取 JSON 文档并从内容创建平面 HashMap。