[英]how to convert rows into columns in spark dataframe using scala
how to convert a dataframe as below?如何转换 dataframe 如下?
a dataframe I have: dataframe 我有:
GROUP![]() |
ITEM![]() |
AMOUNT![]() |
---|---|---|
group1![]() |
item1![]() |
100 ![]() |
group1![]() |
item2![]() |
200 ![]() |
group1![]() |
item3![]() |
300 ![]() |
group2![]() |
item1![]() |
400 ![]() |
group2![]() |
item2![]() |
500 ![]() |
expected result预期结果
GROUP![]() |
ITEM1![]() |
ITEM2![]() |
ITEM3![]() |
---|---|---|---|
group1![]() |
100 ![]() |
200 ![]() |
300 ![]() |
group2![]() |
400 ![]() |
500 ![]() |
You can use pivot您可以使用 pivot
val pivotDF = df.groupBy("GROUP").pivot("ITEM").first("AMOUNT")
pivotDF.show()
You can read more about pivot here https://databricks.com/blog/2016/02/09/reshaping-data-with-pivot-in-apache-spark.html您可以在此处阅读有关 pivot 的更多信息https://databricks.com/blog/2016/02/09/reshaping-data-with-pivot-in-apache-spark.ZFC35FDC70D5FC69D2698883A
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.