Spark UI available on Dataproc Cluster?
Question
希望与Dataproc中默认集群上的传统Spark Web GUI进行交互。
1 answers
solution1
3 ACCPTED 2017-05-30 09:04:02
This can be done by creating a SSH tunnel to the Dataproc master node. By using a SOCKS proxy you can then access all the applications running on YARN including your Spark sessions.
This guide will walk you through in detail: Dataproc Cluster web interfaces
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.
Related Question
How do I get a spark job to use all available resources on a Google Cloud DataProc cluster?
Dataproc; Spark job fails on Dataproc Spark cluster, but runs locally
Where is the Spark UI on Google Dataproc?
Spinning up a Dataproc cluster with Spark BigQuery Connector
Dataproc: Spark in Cluster Mode from a Notebook
Spark Application Failing when deployed on Dataproc cluster
Submit Spark Streaming job via DataProc UI
Spark UI on Google Dataproc: numbers interpretation
Dataproc cluster runs a maximum of 5 jobs in parallel, ignoring available resources
Spark-submit in dataproc cluster generating java.lang.ClassNotFoundException:
Related Tags