Has anyone tried to parallelize multiple imputation in 'mice' package?

Question

I'm aware of the fact that Amelia R package provides some support for parallel multiple imputation (MI) . However, preliminary analysis of my study's data revealed that the data is not multivariate normal , so, unfortunately, I can't use Amelia . Consequently, I've switched to using mice R package for MI, as this package can perform MI on data that is not multivariate normal.

Since the MI process via mice is very slow (currently I'm using AWS m3.large 2-core instance), I've started wondering whether it's possible to parallelize the procedure to save processing time. Based on my review of mice documentation and the corresponding JSS paper, as well as mice 's source code, it appears that currently the package doesn't support parallel operations. This is sad, because IMHO the MICE algorithm is naturally parallel and, thus, its parallel implementation should be relatively easy and it would result in a significant economy in both time and resources.

Question: Has anyone tried to parallelize MI in mice package, either externally (via R parallel facilities), or internally (by modifying the source code) and what are results, if any? Thank you!

Answer 1

Recently, I've tried to parallelize multiple imputation (MI) via mice package externally, that is, by using R multiprocessing facilities, in particular parallel package, which comes standard with R base distribution. Basically, the solution is to use mclapply() function to distribute a pre-calculated share of the total number of needed MI iterations and then combine resulting imputed data into a single object. Performance-wise , the results of this approach are beyond my most optimistic expectations: the processing time decreased from 1.5 hours to under 7 minutes (!). That's only on two cores. I've removed one multilevel factor, but it shouldn't have much effect. Regardless, the result is unbelievable!