I am unable to use dplyrXdf in hdinsights due to old libraries

Question

I wrote a script using RevoScaleR and dplyrXdf, to my surprise when using HDInsights (Microsoft Azure managed Spark cluster service) I get an installation of R 3.3.3 and I can't install dplyrXdf, neither is the package in the repository nor can I install from git using devtools, I managed to get it installed once updating every single dependence from it's respective github repository but this is madness, took me hours... The biggest issue seems to be dplyr 0.5 which is the latest avaiable package for this service (current CRAN package is 0.7.4) Am I doing something wrong? maybe something in provisioning (like selecting the wrong type of cluster)? I can not believe MS would put so much work in R and not update it's cluster service, I must be missing something here.

Answer 1

You can install all dependencies rather quickly - it took me about 20 minutes. Just look at the error messages and install the packages stated. I needed only these ones