Efficient way to apply a function to a list of lists

Question

I have 80 lists for the project in question. Each list is a list of length 1000. I'd like to run a function on each one (each of the 1000), and assign the results back to the original object. The total data is over 150 gigs so I want to make sure this is most efficient before running it on the actual data. Is this trivial example the best way to do what I need?

# my actual function is obviously more complicated.
# But let's say the goal is to keep 2/5 items in each list
trivial <- function(foo) {
keep <- c("S1", "S2")
foo[which(keep %in% names(foo))]
}

sublist <- replicate(5, as.list(1:5), simplify=FALSE)
names(sublist) <- paste0("S", 1:5)
eachlist <- replicate(5, sublist, simplify = F)
a1 <- a2 <- a3 <- a4 <- a5 <- eachlist

# To clarify the layout
length(a1)
[1] 5
> length(a1[[1]])
[1] 5
> names(a1[[1]])
[1] "S1" "S2" "S3" "S4" "S5"
# I need to drop S3-S5 from each of 5 sublists of a1.
# Now I'd like to repeat this for all 80 lists named a[0-9].


# all the objects have a pattern sometextNUMBER. This list is 
# just the names of all the lists.
listz <-  as.list(ls(pattern="[a-z][0-9]"))
> listz
[[1]]
[1] "a1"

[[2]]
[1] "a2"

[[3]]
[1] "a3"

[[4]]
[1] "a4"

[[5]]
[1] "a5"
# I don't need anything returned, just for a1-a80 updated such that
# in each sublist, 3 of 5 items are dropped.

# This works fine, but my concern now is just scaling this up.
l_ply(listz, function(x){
     assign(as.character(x), llply(get(x), trivial), envir = .GlobalEnv)
    })

Answer 1

You could loop over the list of names, using substitute() and eval() to first construct and then execute the expressions you'd (not!) like to type individually at the command line:

objNames <- ls(pattern="[a-z][0-9]")

for(objName in objNames) {
    expr <- 
        substitute({
            OBJ <- lapply(OBJ, function(X) X[names(X) %in% c("S1", "S2")])
            }, list(OBJ = as.symbol(objName)))
    eval(expr)
}

Answer 2

This is a good use-case for rapply :

listz <- replicate(5, as.list(1:5), simplify=FALSE)
fun <- function(x) x*10
out <- rapply(listz, fun, how="replace")