How to convert a dataFrame to a list of Lists?

Question

I am trying to use spark_apply to apply an R function to a Spark DataFrame. In my initial problem I have a list of lists that I have convert to an R dataframe that I will be able to convert to a Spark DataFrame.

But in my R function I need to use a list of lists and I do not know how to get back my initial list of lists from the dataframe.

I have already find how to convert my dataFrame to a list :

lapply(seq(along = rownames(testP)),
         function(i) as.list(testP[i, ])) 

I finally have a list of 24 lists but I am looking for 3 lists of 8 lists.

Thank you for your help !

Answer 1

Maybe you can inspire yourself in the following code. Not a solution, since there is no data set example in the question.

testP <- data.frame(X = rnorm(24), A = sample(letters[1:4], 24, TRUE))

f <- rep(c(1, rep(0, nrow(testP)/3 - 1)), 3)
sp <- split(testP, cumsum(f))
res <- lapply(sp, function(DF) apply(DF, 1, as.list))

str(res)