使用R中的数据帧的两个向量的元素操作

Question

My first question here: how to apply an efficient routine that iterates values of two vectors (pairwise) of a given data frame?

To be more specific, consider the following example using the following data frame:

df0 <- data.frame(matrix(c(1,2,2,3,1,3,0.4,0.2,0.2,0.1,0.4,0.1),nrow=6,ncol=2))
colnames(df0) <- c("value","frequency")

The first column is a real value and the second column is a frequency (or weights). NOTICE: the weights have to be strictly positive, they might be repeated, they not necessarily add up to one (because of repetition).

I am performing the following LOOP to calculate my function P. This P is supposed to be a number between 0 and 1.

# Define two parameters
K = 1/2
alpha = 0

# LOOP
mattemp <- matrix(,nrow=length(df0$value), ncol=length(df0$value))

for(i in 1:length(df0$value)) {
  for(j in 1:length(df0$value)) {

    mattemp[i,j] <- df0$frequency[i]^(1+alpha) * df0$frequency[j] * abs(df0$value[i]-df0$value[j])

    P <- K * sum(mattemp)
  }
}

Basically, my function P is calculating:

P = K * (0.4^alpha * 0.2 * |1-2| + 0.4^alpha * 0.1 * |1-3| + ...

This code works perfectly well as long as the matrix is small.

However, I am trying to implement this routine for a big matrix (5400 x 5400) and this LOOP does not seem to find an end.

I already tried to loop it using a foreach command (using %dopar% ), but it does not work as well.

Is there a smart and concise routine that R can handle??? It does not need to follow the above structure, as long as it is efficient.

Thank you very much

Answer 1

Try:

df$nval <- (df0$value - mean(df0$value)) / sd(df0$value)
ij <- combn(nrow(df0), 2)
foo <- sum(df0$frequency[ij[1, ]] ^ (1 + alpha) * df0$frequency[ij[2, ]] * abs(df0$nval[ij[1, ]] - df0$nval[ij[2, ]]))
P <- K*2*sum(foo)

Reasoning : Basically you are testing every possible permutation between frequencies and normalized values. We use combn to create half of those. We then just vectorize the whole thing. Since combn only gives unique combinations, we need to multiply by 2. [Keep in mind that we don't need the values on the diagonal, as abs(df0$value[i] - df0$value[i]) is equal to 0 , and we are only missing cases where i=j and j=i , so that's why we multiply by 2.] We then multiply by K and get P.

It's not clear how you want to normalize, so I just substracted the mean and divided that by the standard deviation. If you meant something else, you yourself can change it accordingly.

Edit1 : Big thanks to @alexis_laz for finding a mistake and suggesting improvements that almost double the speed!

Edit2 : Adjusted script to fit changed requirements.