R 错误：java.lang.OutOfMemoryError：Java 堆空间

Question

I am trying to connect R to Teradata to pull data directly into R for analysis. However, I am getting the error of,

Error in .jcall(rp, "I", "fetch", stride, block) :
  java.lang.OutOfMemoryError: Java heap space

I have tried to set up my R options to increase the max heap size of JVM by doing:

options(java.parameters = "-Xmx8g")

I have also tried to initialize java parameters with rJava function .jinit as: .jinit(parameters="-Xmx8g") . But still failed.

The calculated size of the data should be approximately 3G (actually less than 3G).

Answer 1

You need to make sure you're allocating additional memory before loading rJava or any other packages. Wipe the environment first (via rm(list = ls()) ), restart R/Rstudio if you must, and modify the options at the beginning of your script.

options(java.parameters = "-Xmx8000m")

See for example https://support.snowflake.net/s/article/solution-using-r-the-following-error-is-returned-javalangoutofmemoryerror-gc-overhead-limit-exceeded

Answer 2

I somehow had this problem in a not reproducible manner, partly solved it with -Xmx8g but run in to problems randomly.

I now found an option with a different garbage collector by using

options(java.parameters = c("-XX:+UseConcMarkSweepGC", "-Xmx8192m"))
library(xlsx)

at the beginning of the script and before any other package is loaded since other packages can load some java things by themselves and the options have to be set before any Java is loaded.

So far, the problem didn't occurred again.

Only sometimes in a long session it can still happen. But in this case a session restart normally solves the problem.

Answer 3

Running the following two lines of code (before any packages are loaded) worked for me on a Mac:

options(java.parameters = c("-XX:+UseConcMarkSweepGC", "-Xmx8192m"))
gc()

This essentially combines two proposals previously posted herein: Importantly, only running the first line alone (as suggested by drmariod) did not solve the problem in my case. However, when I was additionally executing gc() just after the first line (as suggested by user2961057) the problem was solved.

Should it still not work, restart your R session, and then try (before any packages are loaded) instead options(java.parameters = "-Xmx8g") and directly after that execute gc() . Alternatively, try to further increase the RAM from "-Xmx8g" to eg "-Xmx16g" (provided that you have at least as much RAM).

EDIT: Further solutions: While I had to use the rJava for model estimations in R (explaining y from a large number of X ), I kept receiving the above 'OutOfMemory' Errors even if I scaled up to "-Xmx60000m" (the machine I am using has 64 GB RAM). The problem was that some model specifications were simply too big (and would have required even more RAM). One solution that may help in this case is scaling the size of the problem down (eg by reducing the number of X's in the model), or – if possible – splitting the problem into independent pieces, estimating each separately, and putting those pieces together again.

Answer 4

I added garbage collection and that solved the issue for me. I am connecting to Oracle databases using RJDBC.
simply add gc()