Large subset selection from data.frame

Question

I have a huge dataset:

library(gtools)
a<-permutations(2,20,v=c(0,1),repeats.allowed=TRUE)
a<-as.data.frame(a)

And I have a matrix:

set.seed(123)
b<-replicate(5,sample(1:20,5, replace=T))
b<-t(b)

For each row of 'a' I would like to select the columns specified by each column in 'b'

to do so I run the following:

for (i in 1:nrow(a)) sapply(1:N, function(y) a[i,c(as.vector(b[,y]))]) 

As a result I want a matrix or data.frame with the selected columns from 'a' for each row of 'a'

The problem is that this process is extremely slow. I would like to know if there is a faster way to do this.

The above example is do show how slow the process is. Here is a smaller example:

 library(gtools)
 a<-permutations(2,5,v=c(0,1),repeats.allowed=TRUE)
 a<-as.data.frame(a)



 set.seed(123)
  b<-replicate(5,sample(1:5,5, replace=T))
  b<-t(b)

here's what I want step by step:

1. select the i-th row in `'a'`
2. select the y-th column in `'b'`

3.select those elements in the first row of `'a'` that are specified by the first column in `'b'`

4. Repeat 2. and 3. until all columns of 'b' have been used.

This is done using:

sapply(1:N, function(y) a[i,c(as.vector(b[,y]))]) 

1. Repeat 1-4 for each row in 'a'

This is done by adding the for loop:

for (i in 1:nrow(a)) sapply(1:ncol(b), function(y) a[i,c(as.vector(b[,y]))]) 

Answer 1

Using a smaller subset of a

 a1 <- a[1:22,]
 a2 <- as.matrix(a1[,c(b)])

 res1 <- lapply(split(a2, row(a2)), function(x) { matrix(x,ncol=ncol(b))})

Or by keeping it in array

 arr1 <- array(t(a2), dim=c(5,5,22))

res1[[22]]
#      [,1] [,2] [,3] [,4] [,5]
#[1,]    0    1    0    1    0
#[2,]    0    0    1    0    0
#[3,]    1    0    0    0    0
#[4,]    1    0    0    0    1
#[5,]    1    0    0    1    0

arr1[,,22]
#      [,1] [,2] [,3] [,4] [,5]
# [1,]    0    1    0    1    0
# [2,]    0    0    1    0    0
# [3,]    1    0    0    0    0
# [4,]    1    0    0    0    1
# [5,]    1    0    0    1    0