Google Cloud Dataproc migration to Spark 1.6.0

Question

Will Google Dataproc start using Spark 1.6.0 anytime soon ? I'm creating a cluster using this command:

gcloud beta dataproc clusters create <cluster-name>

and it defaults to using Spark 1.5.2.

Thanks.

Answer 1

您可能会在接下来的几周内看到包含Spark 1.6.x的Google Cloud Dataproc的新映像版本。

Answer 2

https://cloud.google.com/dataproc/dataproc-versions lists the available versions. Currently you have a choice of Spark 1.5.0 or Spark 1.5.2 (the default).