I am trying to run a very simple model with rstan in batch mode and have some errors. I do not have any errors when I try to run the same code not in batch mode.
The community from STAN Forums helped me to understand that the issue is that Rcpp instead of STAN is not working properly on the server of my university. We reached this understanding from a test trying to run a very simple Rcpp code in batch mode and not in batch mode. The Rcpp code is:
### Libraries
library(Rcpp)
Rcpp::sourceCpp(code='
#include <Rcpp.h>
// [[Rcpp::export]]
int fibonacci(const int x) {
if (x == 0) return(0);
if (x == 1) return(1);
return (fibonacci(x - 1)) + fibonacci(x - 2);
}',
verbose = TRUE,
rebuild = TRUE
)
The results are below.
Not in batch mode:
Generated R functions
-------------------------------------------------------
`.sourceCpp_1_DLLInfo` <- dyn.load('/tmp/RtmpDNL6of/sourceCpp-x86_64-pc-linux-gnu-1.0.7/sourcecpp_178b21d435d91/sourceCpp_4.so')
fibonacci <- Rcpp:::sourceCppFunction(function(x) {}, FALSE, `.sourceCpp_1_DLLInfo`, 'sourceCpp_1_fibonacci')
rm(`.sourceCpp_1_DLLInfo`)
Building shared library
--------------------------------------------------------
DIR: /tmp/RtmpDNL6of/sourceCpp-x86_64-pc-linux-gnu-1.0.7/sourcecpp_178b21d435d91
/hpc/apps/R/4.1.2/lib64/R/bin/R CMD SHLIB --preclean -o 'sourceCpp_4.so' 'file178b25628f5ea.cpp'
g++ -std=gnu++14 -I"/hpc/apps/R/4.1.2/lib64/R/include" -DNDEBUG -I"/hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include" -I"/tmp/RtmpDNL6of/sourceCpp-x86_64-pc-linux-gnu-1.0.7" -I/hpc/apps/zlib/1.2.8/include -I/hpc/apps/pcre/8.38/include -I/hpc/apps/bzip2/1.0.6/include -I/hpc/apps/curl/7.54.1/include -I/hpc/apps/cairo/1.14.12/include -fpic -g -O2 -c file178b25628f5ea.cpp -o file178b25628f5ea.o
g++ -std=gnu++14 -shared -L/hpc/apps/R/4.1.2/lib64/R/lib -L/hpc/apps/zlib/1.2.8/lib -L/hpc/apps/pcre/8.38/lib -L/hpc/apps/bzip2/1.0.6/lib -L/hpc/apps/curl/7.54.1/lib -L/hpc/apps/cairo/1.14.12/lib -o sourceCpp_4.so file178b25628f5ea.o -L/hpc/apps/R/4.1.2/lib64/R/lib -lR
In batch mode:
Generated R functions
-------------------------------------------------------
`.sourceCpp_1_DLLInfo` <- dyn.load('/localscratch/250948.1.ragatkolab.q/RtmpDlK5Sy/sourceCpp-x86_64-pc-linux-gnu-1.0.7/sourcecpp_142bd68d660f1/sourceCpp_2.so')
fibonacci <- Rcpp:::sourceCppFunction(function(x) {}, FALSE, `.sourceCpp_1_DLLInfo`, 'sourceCpp_1_fibonacci')
rm(`.sourceCpp_1_DLLInfo`)
Building shared library
--------------------------------------------------------
DIR: /localscratch/250948.1.mylab.q/RtmpDlK5Sy/sourceCpp-x86_64-pc-linux-gnu-1.0.7/sourcecpp_142bd68d660f1
/hpc/apps/R/4.1.2/lib64/R/bin/R CMD SHLIB -o 'sourceCpp_2.so' 'file142bd64ebe902.cpp'
g++ -std=gnu++14 -I"/hpc/apps/R/4.1.2/lib64/R/include" -DNDEBUG -I"/hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include" -I"/localscratch/250948.1.ragatkolab.q/RtmpDlK5Sy/sourceCpp-x86_64-pc-linux-gnu-1.0.7" -I/hpc/apps/zlib/1.2.8/include -I/hpc/apps/pcre/8.38/include -I/hpc/apps/bzip2/1.0.6/include -I/hpc/apps/curl/7.54.1/include -I/hpc/apps/cairo/1.14.12/include -fpic -g -O2 -c file142bd64ebe902.cpp -o file142bd64ebe902.o
In file included from /hpc/apps/gcc/9.1.0/include/c++/9.1.0/x86_64-pc-linux-gnu/bits/c++config.h:524,
from /hpc/apps/gcc/9.1.0/include/c++/9.1.0/cmath:41,
from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp/platform/compiler.h:100,
from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp/r/headers.h:66,
from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/RcppCommon.h:30,
from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp.h:27,
from file142bd64ebe902.cpp:2:
/hpc/apps/gcc/9.1.0/include/c++/9.1.0/x86_64-pc-linux-gnu/bits/os_defines.h:39:10: fatal error: features.h: No such file or directory
39 | #include <features.h>
| ^~~~~~~~~~~~
compilation terminated.
make: *** [file142bd64ebe902.o] Error 1
Error in Rcpp::sourceCpp(code = "\n #include <Rcpp.h>\n\n // [[Rcpp::export]]\n int fibonacci(const int x) {\n if (x == 0) return(0);\n if (x == 1) return(1);\n return (fibonacci(x - 1)) + fibonacci(x - 2);\n }", :
Error 1 occurred building shared library.
WARNING: The tools required to build C++ code for R were not found.
Please install GNU development tools including a C++ compiler.
Execution halted
Then, they asked me to modify my makevars file - which I did using the R-package usethis::edit_r_makevars()
- adding the lines:
CXXFLAGS += -I"/hpc/apps/gcc/9.1.0/include/c++/9.1.0/parallel/"
CXX14FLAGS += -I"/hpc/apps/gcc/9.1.0/include/c++/9.1.0/parallel/"
Once I did this modification (I hope I did it correctly), my error message change to:
Generated extern "C" functions
--------------------------------------------------------
#include <Rcpp.h>
#ifdef RCPP_USE_GLOBAL_ROSTREAM
Rcpp::Rostream<true>& Rcpp::Rcout = Rcpp::Rcpp_cout_get();
Rcpp::Rostream<false>& Rcpp::Rcerr = Rcpp::Rcpp_cerr_get();
#endif
// fibonacci
int fibonacci(const int x);
RcppExport SEXP sourceCpp_1_fibonacci(SEXP xSEXP) {
BEGIN_RCPP
Rcpp::RObject rcpp_result_gen;
Rcpp::RNGScope rcpp_rngScope_gen;
Rcpp::traits::input_parameter< const int >::type x(xSEXP);
rcpp_result_gen = Rcpp::wrap(fibonacci(x));
return rcpp_result_gen;
END_RCPP
}
Generated R functions
-------------------------------------------------------
`.sourceCpp_1_DLLInfo` <- dyn.load('/localscratch/251578.1.mylablab.q/RtmpCHyEAd/sourceCpp-x86_64-pc-linux-gnu-1.0.7/sourcecpp_1e50364a03315/sourceCpp_2.so')
fibonacci <- Rcpp:::sourceCppFunction(function(x) {}, FALSE, `.sourceCpp_1_DLLInfo`, 'sourceCpp_1_fibonacci')
rm(`.sourceCpp_1_DLLInfo`)
Building shared library
--------------------------------------------------------
DIR: /localscratch/251578.1.mylablab.q/RtmpCHyEAd/sourceCpp-x86_64-pc-linux-gnu-1.0.7/sourcecpp_1e50364a03315
/hpc/apps/R/4.1.2/lib64/R/bin/R CMD SHLIB -o 'sourceCpp_2.so' 'file1e50325e7686.cpp'
g++ -std=gnu++14 -I"/hpc/apps/R/4.1.2/lib64/R/include" -DNDEBUG -I"/hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include" -I"/localscratch/251578.1.mylablab.q/RtmpCHyEAd/sourceCpp-x86_64-pc-linux-gnu-1.0.7" -I/hpc/apps/zlib/1.2.8/include -I/hpc/apps/pcre/8.38/include -I/hpc/apps/bzip2/1.0.6/include -I/hpc/apps/curl/7.54.1/include -I/hpc/apps/cairo/1.14.12/include -fpic -g -O2 -I"/hpc/apps/gcc/9.1.0/include/c++/9.1.0/parallel/" -c file1e50325e7686.cpp -o file1e50325e7686.o
In file included from /hpc/apps/gcc/11.1.0/include/c++/11.1.0/x86_64-pc-linux-gnu/bits/c++config.h:571,
from /hpc/apps/gcc/11.1.0/include/c++/11.1.0/cmath:41,
from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp/platform/compiler.h:100,
from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp/r/headers.h:66,
from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/RcppCommon.h:30,
from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp.h:27,
from file1e50325e7686.cpp:2:
/hpc/apps/gcc/11.1.0/include/c++/11.1.0/x86_64-pc-linux-gnu/bits/os_defines.h:44:19: error: missing binary operator before token "("
44 | #if __GLIBC_PREREQ(2,15) && defined(_GNU_SOURCE)
| ^
/hpc/apps/gcc/11.1.0/include/c++/11.1.0/x86_64-pc-linux-gnu/bits/os_defines.h:52:19: error: missing binary operator before token "("
52 | #if __GLIBC_PREREQ(2, 27)
| ^
In file included from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp/platform/compiler.h:100,
from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp/r/headers.h:66,
from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/RcppCommon.h:30,
from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp.h:27,
from file1e50325e7686.cpp:2:
/hpc/apps/gcc/11.1.0/include/c++/11.1.0/cmath:45:15: fatal error: math.h: No such file or directory
45 | #include_next <math.h>
| ^~~~~~~~
compilation terminated.
make: *** [file1e50325e7686.o] Error 1
Error in Rcpp::sourceCpp(code = "\n #include <Rcpp.h>\n\n // [[Rcpp::export]]\n int fibonacci(const int x) {\n if (x == 0) return(0);\n if (x == 1) return(1);\n return (fibonacci(x - 1)) + fibonacci(x - 2);\n }", :
Error 1 occurred building shared library.
WARNING: The tools required to build C++ code for R were not found.
Please install GNU development tools including a C++ compiler.
Execution halted
My knowledge in Linux is limited and I am afraid that my question is out of scope for STAN Forums. Any thoughts how to proceed?
I fear your problem is still local. On the computing you use, someone may have set up R in way that makes it differ between interactive R use, and what you call batch use.
But that is non-standard, and at your end. In general, it just works. Using a minimally modified version of your program (which I still find confusing: it loads doParallel
, registers cores but... triggers to parallel code -- anyway) it works just fine:
edd@rob:~/git/stackoverflow/70470842(master)$ Rscript answer.R
Loading required package: foreach
Loading required package: iterators
Loading required package: parallel
Fib(10) is 55
edd@rob:~/git/stackoverflow/70470842(master)$
edd@rob:~/git/stackoverflow/70470842(master)$ R -s -f answer.R
Loading required package: foreach
Loading required package: iterators
Loading required package: parallel
Fib(10) is 55
edd@rob:~/git/stackoverflow/70470842(master)$
edd@rob:~/git/stackoverflow/70470842(master)$ r answer.R
Loading required package: foreach
Loading required package: iterators
Loading required package: utils
Loading required package: parallel
Fib(10) is 55
edd@rob:~/git/stackoverflow/70470842(master)$
library(doParallel) # Library
ncores <- 5 # Simulation parameters
n_sim <- 5
registerDoParallel(ncores)
Rcpp::sourceCpp(code='
#include <Rcpp.h>
// [[Rcpp::export]]
int fibonacci(const int x) {
if (x == 0) return(0);
if (x == 1) return(1);
return (fibonacci(x - 1)) + fibonacci(x - 2);
}'
)
cat("Fib(10) is ", fibonacci(10), "\n")
For future reference, I interacted with IT team from my institution after feedback received here and STAN Forums, and they finally found a solution. In their words: "Compute nodes aren't normally used for development. So certain development packages are not installed by default. We installed "glibc-devel" and "glibc-headers" on the compute nodes."
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.