使用NA输入的Rcpp R崩溃/中止

Question

I want to process two raster images (Ra and Rb), with Ra is the pixel value itself and Rb the values of its neighbors. taking sum as an example, assuming an 3*3 neighbors, for each pixel in Ra I will add its value to values of the neighbor pixels in Rb, an finally i will get another image.

The R raster package provides a focal function, which works only on one image input, i tried to modify the C++ code ( enter link description here ) to accept two image input using Rcpp. The modified code works well if there is no missing values in the input image of Rb. However, R always aborts if there is NA in Rb. Specifically, abort at the second or third test. it may be similar to this post . however, it did not crash if no NA in the input Rb. It seems i did not handle NA correctly. I do not have deep knowledge on C++, Can somebody help me check this?

here is my cpp file:

#include <Rcpp.h>
#include <R.h>
#include <Rinternals.h>
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <Rmath.h>
#include "Rdefines.h"
#include "R_ext/Rdynload.h"

using namespace Rcpp;
// [[Rcpp::export]]
NumericVector focal_quantile(NumericVector xd, int ngbb, NumericVector sf) {
  //the imges are transfered to vector, ngbb is the size of the window  
  R_len_t i, j, k, q;
  int wrows = ngbb;
  int wcols = ngbb;
  int wn = wrows * wcols;

  int nrow = 6;//the input raste has 6 rows
  int ncol = 7;//the input raste has 7 cols

  int n = nrow * ncol;
  NumericVector xans(n);
  NumericVector xx(wn);

  int wr = floor(wrows / 2);
  int wc = floor(wcols / 2);

  int nwc = ncol - wc - 1;
  int col = 0;

  // first rows
  for (i = 0; i < ncol*wr; i++) {// the first row, the resutl is set as NA as the neighbor does not have nine values   
    xans[i] = R_NaReal; 
  }

  for (i = ncol*wr; i < (ncol * (nrow-wr)); i++) {//start from the second row
    col = i % ncol;
    if ((col < wc) | (col > nwc)) {//the first pixel of the second is also set as NA
      xans[i] = R_NaReal;
    } else {// to get the nine values in the 3*3 windows
      q = 0;
      for (j = -wr; j <= wr; j++) {
        for (k = -wc; k <= wc; k++) {
          xx[q] = xd[j * ncol + k + i]; 
          q++;
        }
      }
      xx = na_omit(xx);
      int n_qt = xx.size();
      if (n_qt > 0){//
        xans[i]=sum(xx)+100*sf[i];// here is the calculation, my goal is more complicated than this example
      } else {
        xans[i] = R_NaReal;//R_NaReal
      }

    }
  }
  // last rows
  for (i = ncol * (nrow-wr); i < n; i++) {  
    xans[i] = R_NaReal;
  }
  return(xans);
}

Then compile it using sourceCpp

generate example data to test it

  rr=raster(nrow=6,ncol=7)## example for Ra
  projection(rr)="+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84"
  rr[]=(2:43)*10
  rrqt=rr/43 ## example for Rb
  ##it works fine, if there is no NA in Ra
  #rr[1:10]=NA #window of global enviornment is refleshing and then aborts with such NAs 
  focal_quantile(rr[],3,rrqt[])

Example results

 [1]       NA       NA       NA       NA       NA       NA       NA       NA 118918.6 130810.5 142702.3 154594.2 166486.0       NA       NA
[16] 202161.6 214053.5 225945.3 237837.2 249729.1       NA       NA 285404.7 297296.5 309188.4 321080.2 332972.1       NA       NA 368647.7
[31] 380539.5 392431.4 404323.3 416215.1       NA       NA       NA       NA       NA       NA       NA       NA

resulted NA is acceptable as there are not nine values in the windows. For such example, i change the values of raster rr (with no NA). it works smoothly. when I introduce NA in rr, for example the sixth row of the codes above. the Global environment window is refreshing and Rstudio aborts.

the session information is

R version 3.3.0 (2016-05-03)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] Rcpp_0.12.11 raster_2.5-8 sp_1.2-3    

loaded via a namespace (and not attached):
[1] rgdal_1.2-5     tools_3.3.0     grid_3.3.0      lattice_0.20-35

Thank you very much!

Answer 1

First of all, you should only be using the #include <Rcpp.h> statement. The other headers you are adding are not needed or already included within Rcpp.h .

---

Secondly, the correct way to reference the NA value for NumericVector s within Rcpp is to use the NA_REAL not R's R_NaReal .

---

Thirdly, you have an out of bounds error. If you switch the parentheses from [] to () you have bounds detection. The error on Rcpp 0.12.11 is:

"Index out of bounds: [index=3; extent=3]."

As a result, this is creating an "Undefined Behavior" (UB) that triggers the crash of RStudio.

The problematic line is:

xx(q) = xd(j * ncol + k + i); 
^^^^^

Now, you might say this doesn't make sense as the length of xx should never be 3. However, the reason this line is problematic is because you are changing the values that are found in xx when you drop the NA values with:

xx = na_omit(xx);

You should really declare a new xy vector if this is the aim or update the constants to ensure the out of bounds error is avoided.

---

Implementation

#include <Rcpp.h>

// [[Rcpp::export]]
Rcpp::NumericVector focal_quantile(Rcpp::NumericVector xd,
                                   int ngbb,
                                   Rcpp::NumericVector sf) {
  //the imges are transfered to vector, ngbb is the size of the window  
  R_len_t i, j, k, q;
  int wrows = ngbb;
  int wcols = ngbb;
  int wn = wrows * wcols;

  int nrow = 6;//the input raste has 6 rows
  int ncol = 7;//the input raste has 7 cols

  int n = nrow * ncol;
  Rcpp::NumericVector xans(n);
  Rcpp::NumericVector xx(wn);

  int wr = floor(wrows / 2);
  int wc = floor(wcols / 2);

  int nwc = ncol - wc - 1;
  int col = 0;

  // first rows
  for (i = 0; i < ncol*wr; i++) {// the first row, the resutl is set as NA as the neighbor does not have nine values   
    xans[i] = NA_REAL; 
  }

  for (i = ncol*wr; i < (ncol * (nrow-wr)); i++) {//start from the second row
    col = i % ncol;
    if ((col < wc) | (col > nwc)) {//the first pixel of the second is also set as NA
      xans[i] = NA_REAL;
    } else {// to get the nine values in the 3*3 windows
      q = 0;
      for (j = -wr; j <= wr; j++) {
        for (k = -wc; k <= wc; k++) {
          xx[q] = xd[j * ncol + k + i]; 
          q++;
        }
      }
      Rcpp::NumericVector xx_subset = na_omit(xx);
      int n_qt = xx_subset.size();
      if (n_qt > 0){//
        xans[i]=sum(xx_subset)+100*sf[i];// here is the calculation, my goal is more complicated than this example
      } else {
        xans[i] = NA_REAL;//NA_REAL
      }

    }
  }

  // last rows
  for (i = ncol * (nrow-wr); i < n; i++) {  
    xans[i] = NA_REAL;
  }
  return(xans);
}

Test case:

library("raster")
rr = raster(nrow=6,ncol=7)## example for Ra
projection(rr) = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84"
rr[] = (2:43)*10
rrqt = rr/43 ## example for Rb
rr[1:10] = NA 
focal_quantile(rr[],3,rrqt[])

Output:

 [1]        NA        NA        NA        NA        NA        NA        NA        NA  742.5581  915.8140 1099.0698 1292.3256
[13] 1375.5814        NA        NA 1625.3488 1828.6047 2041.8605 2265.1163 2378.3721        NA        NA 2718.1395 2831.3953
[25] 2944.6512 3057.9070 3171.1628        NA        NA 3510.9302 3624.1860 3737.4419 3850.6977 3963.9535        NA        NA
[37]        NA        NA        NA        NA        NA        NA

---

Side note

If you look at the code you are trying to translate, note that there is an naonly part followed by na components. So, the translation is not necessarily 1-1.