R recursive function or loop in loop

Question

simple problem. I want to check if the difference of two points (i, j) is greater than a threshold (diff). If the difference between the points exceeds the threshold the index should be returned and the next distance is measured but from the new datapoint. It is a simple cutofffilter where all datapoints under a predefined threshold are filtered. The only trick is, that the measurement is performed from always the "last" point (that was "far enough away" from the point before).

I first wrote it as two nested loops like:

x <- sample(1:100)
for(i in 1:(length(x)-1)){
      for(j in (i+1):length(x)){
        if(abs(x[i] - x[j]) >= cutoff) { 
          print(j)
          i <- j  # set the index to the current datapoint
          break }
      }}

This solution is kind of intuitive. But does not work proper. I think the assignment of i and j is not valid. The first loop just ignores to jump and loops through all datapoints.

Well, I did not want to waste time with debugging and just thought I can do the same with a recursive function. So I wrote it like:

checkCutOff.f <- function(x,cutoff,i = 1) {
  options(expressions=500000)
  # Loops through the data and comperes the temporally fixed point 'i with the looping points 'j
  for(j in (i+1):length(x)){
    if( abs(x[i] - x[j]) >= cutoff ){
      break
    }
  }

  # Recursive function to update the new 'i - stops at the end of the dataset
  if( j<length(x) ) return(c(j,checkCutOff.f(x,cutoff,j))) 
  else return(j)
}
 x<-sample(1:100000)
 checkCutOff.f(x,1)

This code works. But I get a stack overflow with big datasets. That's why I ask myself if this code is efficient. For me is increasing limits etc. always a hint for inefficient code...

So my question is: What kind of solution is really efficient? Thanks!

Answer 1

You should avoid growing your return value with c . That's inefficient. Allocate to the maximum size and subset to the needed size in the end.

Note that your function always includes length(x) in your result, which is wrong:

set.seed(42)
x<-sample(1:10)
checkCutOff.f(x, 100)
#[1] 10

Here is an R solution with a loop:

checkCutOff.f1 <- function(x,cutoff) {
  i <- 1
  j <- 1
  k <- 1

  result <- integer(length(x))

  while(j < length(x)) {
    j <- j + 1
    if (abs(x[i] - x[j]) >= cutoff) {
      result[k] <- j
      k <- k + 1
      i <- j
    }
  }
  result[seq_len(k - 1)]
}

all.equal(checkCutOff.f(x, 4), checkCutOff.f1(x, 4))
#[1] TRUE
#the correct solution includes length(x)  here (by chance)

It's easy to translate to Rcpp:

#include <Rcpp.h>
using namespace Rcpp;


// [[Rcpp::export]]
IntegerVector checkCutOff_f1cpp(NumericVector x, double cutoff) {
  int i = 0; 
  int j = 1; 
  int k = 0;
  IntegerVector result(x.size());  
  while(j < x.size()) {
    if (std::abs(x[i] - x[j]) >= cutoff) {
      result[k] = j + 1;
      k++;
      i = j;
    }
    j++;
  }
  result = result[seq_len(k)-1];
  return result;
}

Then in R:

all.equal(checkCutOff.f(x, 4), checkCutOff_f1cpp(x, 4))
#[1] TRUE

Benchmarks:

library(microbenchmark)
y <- sample(1:1000)

microbenchmark(
  checkCutOff.f(y, 4),
  checkCutOff.f1(y, 4),
  checkCutOff_f1cpp(y, 4)
  )

#Unit: microseconds
#                    expr      min        lq       mean   median        uq       max neval cld
#     checkCutOff.f(y, 4) 3665.105 4681.6005 7798.41776 5323.068 6635.9205 41028.930   100   c
#    checkCutOff.f1(y, 4) 1384.524 1507.2635 1831.43236 1769.031 2070.7225  3012.279   100  b 
# checkCutOff_f1cpp(y, 4)    8.765   10.7035   26.40709   14.240   18.0005   587.958   100 a

I'm sure this can be improved further and more testing should be done.