[英]How to dynamically build column name in spark sql withColumn
I have sample data as below.我有如下示例数据。
I would like populate another column exp
based on country
field value.我想根据country
/地区字段值填充另一列exp
。
Something like below像下面的东西
df.withColumn("exp",col(s"exp_$country"))
So that respective country
number can be placed there.这样相应的country
号码就可以放在那里。
But above code errors out saying:但上面的代码错误说:
cannot resolve country无法解析国家
Output I need is Output 我需要的是
Any help appreciated.任何帮助表示赞赏。
You can chain multiple when
expressions from the list of countries:您可以从国家列表中链接多个when
表达式:
val countries = Seq("us", "uk", "ind")
val expCol = countries.foldLeft(lit(null)) { case (acc, country) =>
when(col("country")===country, col(s"exp_$country")).otherwise(acc)
}
val df1 = df.withColumn("exp", expCol)
Or if you prefer creating a map expression country -> exp
from the columns exp_*
than use the map to create exp
column:或者,如果您更喜欢从exp_*
列创建 map 表达式country -> exp
,而不是使用 map 创建exp
列:
val mapCountries = map(
df.columns
.filter(_.startsWith("exp_"))
.flatMap(c => Seq(lit(c.split("_")(1)), col(c))): _*
)
val df1 = df.withColumn("exp", mapCountries(col("country")))
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.