![](/img/trans.png)
[英]How to transform Scala nested map operation to Scala Spark operation?
[英]Spark Scala - How to construct Scala Map from nested JSON?
我有一個嵌套的json數據,其中包含要提取和構造Scala Map的嵌套字段。
這是示例JSON:
"nested_field": [
{
"airport": "sfo",
"score": 1.0
},
{
"airport": "phx",
"score": 1.0
},
{
"airport": "sjc",
"score": 1.0
}
]
我想使用saveToES()並構造一個Scala映射,以如下所示將字段索引到ES索引中:
"nested_field": {
"properties": {
"score": {
"type": "double"
},
"airport": {
"type": "keyword",
"ignore_above": 1024
}
}
}
使用spark.read.json(“ example.json”)將json文件讀入數據框。 在這種情況下構造Scala地圖的正確方法是什么?
謝謝你的幫助!
您可以使用以下示例代碼進行操作
import org.json4s.DefaultFormats
import org.json4s.jackson.JsonMethods.parse
case class AirPortScores(airport: String, score: Double)
case class JsonRulesHandler(airports: List[AirPortScores])
val jsonString: String = """{"airports":[{"airport":"sfo","score":1},{"airport":"phx","score":1},{"airport":"sjc","score":1}]}"""
def loadJsonString(JsonString: String): JsonRulesHandler = {
implicit val formats: DefaultFormats.type = org.json4s.DefaultFormats
parse(JsonString).extract[JsonRulesHandler]
}
val parsedJson: JsonRulesHandler = loadJsonString(jsonString)
parsedJson.airports.foreach(println)//you can select parsedJson.airport or scores
//below ouput
AirPortScores(sfo,1.0)
AirPortScores(phx,1.0)
AirPortScores(sjc,1.0)
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.