SparkR与众不同（在数据块上）

Question

I am new to SparkR, so please forgive if my question is very basic.

I work on databricks and try to get all unique dates of a column of a SparkDataFrame.

When I run:

uniquedays <- SparkR::distinct(df$datadate)

I get the error message:

unable to find an inherited method for function ‘distinct’ for signature ‘"Column"’

On Stack Overflow , I found out that this usually means (If I run isS4(df), it returns TRUE):

That is the type of message you will get when attempting to apply an S4 generic function to an object of a class for which no defined S4 method exists

I also tried to run

uniquedays <- SparkR::unique(df$datadate)

where I get the error message:

unique() applies only to vectors

It feels like, I am missing something basic here. Thank you for your help!

Answer 1

Try this:

library(magrittr)
uniquedays <- SparkR::select(df, df$datadate) %>% SparkR::distinct()