在R或Python中随机化矩阵的最有效方法

Question

I'm working with a numeric matrix M in R which is quite big (11000 rows per 20 columns). On this matrix, I'm performing a lot of correlation tests

=> the function cor.test(M[i,], M[j,], method='spearman') where i and j are two rows from the matrix (all possible combinations are tested).

The problem as you know is that I'm doing too many tests to get a very reliable p-value returned by this test.

My strategy to overcome this limitation would be to generate a new probability distribution by Bootstrap on my matrix M: I would like to get 100 random matrices generated from M to do the multiple correlations on these matrices and choose the right cut-off for the p-value to get a FDR of 5%.

My question is:

1. What is the most efficient way to randomize my matrix?
2. Since it's quite time consumming (I suppose) it could be interresting if the solution could be parallelized.

Thank you in advance for all the usefull answers that you'll provide to me.

Answer 1

In python there is a function random.sample() in module random . If you store M as list of rows, randomly sampling n rows from matrix M without replacement would be like this

M_sample = random.sample(M,n)

However, for bootstrapping, you might want to do random sampling with replacement. To do this, you can use numpy.random.choice() :

import numpy
M_sample = numpy.random.choice(M,n,replace=True)

In R, we use sample() to randomly decide the row indices to take, and then use row access to take the rows from the matrices. Randomly sampling n rows from matrix M without replacement is done as follows:

indices = sample(nrow(M), n,replace=FALSE)
M_sample = M[indices, ]

And for randomly sampling with replacement, replace the first line with this:

indices = sample(nrow(M), n,replace=TRUE)