How to import Large Datasets in Revolution R Enterprise or R Studio?

Question

I'm currently using Revolution R Enterprise Version 7.5 .

I am trying to import a dataset from Excel into RRE . Here is my code:

library("dplyr")
library("XLConnectJars")
library("XLConnect")

dataset1 <- readWorksheetFromFile(file.choose(), sheet=1, startRow=1, Header=T)

When I try to execute this block of code, this is the error I get:

Error: OutOfMemoryError (Java): GC overhead limit exceeded

Is there anyway to get large amounts of data into Revolution R Enterprise ? Maybe some RevoScaleR library function that I can use?

Even help in R Studio will do.

Answer 1

Although I haven't used this particular package ( XLConnect ), the error you are getting is because it imports rJava . You need to change the allocated Java heap size using the following command:

options(java.parameters="-Xmx4g")

That will change it from the default of 512MB to 4GB; you can choose a different value if you like.

You need to run that command BEFORE you load the libraries. If I were you, I would restart your R session, run that command, and then proceed.