How to aggregate values in Spark?
Question
val trans = df.groupBy("userId").agg(collect_list("movieId") as "features")
How do I aggregate other columns in the dataFrame as well? For now It is only aggregating column MovieID.
1 answers
solution1
0 2018-10-04 18:02:55
您可以添加其他聚合，例如以下示例
val trans = df.groupBy("userId").agg(collect_list("movieId") as "features",avg("rating") as "avg_rating"))
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.
Related Question
How to aggregate the values of 2 columns in a spark dataframe
How aggregate by value in apache spark
Spark : aggregate values by a list of given years
Aggregate key-values in order in spark scala
Spark DataFrame, how to to aggregate sequence of columns?
How to aggregate on two columns in Spark SQL
In spark how to use window spec with aggregate functions
How to aggregate the columns dynamically in spark scala?
How to aggregate an array of struct in spark with group by
Spark Scala: Aggregate DataFrame Column Values into a Ordered List
Related Tags