如何在Spark中汇总值？

Question

val trans = df.groupBy("userId").agg(collect_list("movieId") as "features")

如何在dataFrame中汇总其他列？ 目前，它仅聚合MovieID列。

Answer 1

您可以添加其他聚合，例如以下示例

val trans = df.groupBy("userId").agg(collect_list("movieId") as "features",avg("rating") as "avg_rating"))