TryCatch and names Error when running doRedis examples from documentation

Question

I am trying to use doRedis to create an R cluster and parallel process some of my calculations with foreach, but keep running into errors. My code works with foreach %do% but when I try to run on the cluster with %dopar% it fails. I also tried running the example code in the doRedis docs which fails as well. Below is the R console of my main script and a worker:

The main R console:

> require('doRedis')
> registerDoRedis('work')
> getDoParWorkers()
[1] 2
> foreach(j=1:10,.combine=sum,.multicombine=TRUE) %dopar%
+ 4*sum((runif(1000000)^2 + runif(1000000)^2)<1)/10000000
[1] "interrupt: \n"
Error in tryCatchOne(expr, names, parentenv, handlers[[1L]]) : 
  attempt to apply non-function
In addition: Warning message:
In e$fun(obj, substitute(ex), parent.frame(), e$data) :
  Queue length off by 10...correcting

The worker console:

> require('doRedis')
> redisWorker('work')
Waiting for doRedis jobs.
Processing job  3  from queue  work 
Error in names(z) <- w[o] : 
  'names' attribute [69] must be the same length as the vector [68]

I am not sure what is causing the errors but it seems like everything I send to the worker results in this error:

`Error in names(z) <- w[o] : 
   'names' attribute [69] must be the same length as the vector [68]`

Anyone have any ideas how to fix this?

Answer 1

One problem with your example is that you're not using curly-braces around the body of the foreach loop. Because %dopar% and %do% are binary operators, you need to be sure that the entire body of the foreach loop is treated as the right argument to the %dopar% operator. In your case, the body of the foreach loop is simply 4 . By using curly-braces (or parentheses), you can fix that problem:

foreach(j=1:10,.combine=sum,.multicombine=TRUE) %dopar% {
  4*sum((runif(1000000)^2 + runif(1000000)^2)<1)/10000000
}

But the real problem appears to occur when the redis worker calls the redisInfo function to determine the version of the redis server. I think there's an error in redisInfo when the "INFO" output from the server is parsed which causes the worker to fail. My guess is that when you installed a different redis server, it changed the "INFO" output which no longer triggered the bug in redisInfo, thus "fixing" the problem. Hopefully the redisInfo function can be improved so that this error doesn't occur again.

Answer 2

I think I found the issue. I used brew to install redis-server. I reinstalled redis-server without brew and set timeout to 0 in the config file. Now when I run a job the workers output:

`Waiting for doRedis jobs.
 Processing job  3  from queue  works 
 Processing task 10 ... from queue works jobID 3 
 Processing job  3  from queue  works 
 Processing task 100 ... from queue works jobID 3 
 Processing job  3  from queue  works 
 Processing task 13 ... from queue works jobID 3 
 Processing job  3  from queue  works 
 Processing task 14 ... from queue works jobID 3 
 Processing job  3  from queue  works 
 Processing task 16 ... from queue works jobID 3 
 Processing job  3  from queue  works 
 Processing task 18 ... from queue works jobID 3 
 Processing job  3  from queue  works 
 Processing task 2 ... from queue works jobID 3 
 Processing job  3  from queue  works 
 Processing task 21 ... from queue works jobID 3 
 [1] "Empty"`

I get back an empty string but I think that is because there is an error with the way I have written my function and passing it to foreach.