我们如何使用 Scala 在 spark 中添加列值？

Question

I have data like this

+-------------+--------+--------+
|         COl1|    Col2|    COL3|
+-------------+--------+--------+
|A .......... |   56102|   45991|
|B........... |   25336|   23099|
+-------------+--------+--------+

it should be like this

+-------------+--------+--------+
|         COl1|    Col2|    COL3|
+-------------+--------+--------+
|A .......... |   56102|   45991|
|B........... |   25336|   23099|
|Total....... |   58368|   69090|
+-------------+--------+--------+

need a row with Total and the value should be the sum of reaming row in the dataframe.

Answer 1

You can use aggregation functions to compute the sums, and a union to append them at the end of the original df. For it to work, you just need to make sure that the names of the columns coincide.

It would go like this:

val df = Seq(("A", 56102, 45991), ("B",  25336, 23099))
    .toDF("COL1", "COL2", "COL3")

val sums = df.select(lit("Total") as "COL1", sum('COL2) as "COL2", sum('COL3) as "COL3")
df.union(sums).show()
+-----+-----+-----+
| COL1| COL2| COL3|
+-----+-----+-----+
|    A|56102|45991|
|    B|25336|23099|
|Total|81438|69090|
+-----+-----+-----+