R Snowfall - Call a parallel function within parallel function?

Question

I have recently started using the Snowfall package in R. I have it working successfully in quite a complicated implementation, as follows (with the y loop processed in parallel):

increment x from 1:100 {
    increment y from 1:100 {
        increment z from 1:20 { }
        increment q from 1:20 { }
    }
}

I am running this on a 2 or 4 processor computer. In theory, I suppose I could run the x, y, z and q loops all in parallel. so run x counter in parallel, then for each parallel x process, run y in parallel etc.

My question is does this make sense when using so few processors? eg with four processors, the y-loop computations I would imagine will keep the process running at max output (on average 25 on each processor at any one time) and so splitting other parts of the process will not save time.

Answer 1

You should only parallelize the outer loop since you have enough iterations to use all of your cores. Things can get tricky if the number of iterations and cores can vary, but for your problem, parallelizing the other loops can only hurt performance.

I wrote a vignette on running nested loops in parallel: Nesting Foreach Loops . Although you're not using foreach , you may find it helpful.