[英]Exposing Spark Worker (stdout stderr) logs in Kubernetes
I have a Spark Cluster with one Master and 4 Workers running in a 3 Node Kubernetes Cluster. 我有一个Spark集群,其中一个Master和4个Worker在一个3节点Kubernetes集群中运行。 The Spark UI and Master are exposed through Ingress/NodePort and is hence accessible from outside the Kubernetes cluster. Spark UI和Master通过Ingress / NodePort公开,因此可以从Kubernetes集群外部进行访问。
But, the worker ports are not exposed. 但是,工作端口未暴露。 Due to this, the Worker logs (stdout and stderr) are not accessible through the UI. 因此,无法通过UI访问Worker日志(stdout和stderr)。 The logs URL is getting redirected to <Worker1_Pod_IP:8080>
, <Worker2_Pod_IP:8080>
and so on. 日志URL将被重定向到<Worker1_Pod_IP:8080>
和<Worker2_Pod_IP:8080>
,依此类推。
My setup is such that there are two worker pods running on the same machine. 我的设置是在同一台计算机上运行两个工作容器。 So even if I NodePort expose the workers, there will be a conflict in ports as the same port will be assigned for the two workers on the machine. 因此,即使我的NodePort暴露了工作程序,端口也将发生冲突,因为将为计算机上的两个工作程序分配相同的端口。 Spark History server only provides Event Logs and not Worker logs. Spark History服务器仅提供事件日志,而不提供辅助日志。
How can this be solved? 如何解决呢? Is there a way the NodePort value can be dynamically assigned for the workers 有没有一种方法可以为工作人员动态分配NodePort值
I believe you are talking about the SPARK_WORKER_WEBUI_PORT
and not the SPARK_WORKER_PORT
as described below since that its assigned a random port. 我相信您是在谈论SPARK_WORKER_WEBUI_PORT
而不是SPARK_WORKER_PORT
如下所述,因为它分配了一个随机端口。
This is a bit tricky because you can only expose a single port per node. 这有点棘手,因为每个节点只能公开一个端口。 If you have two spark workers per node you could create two deployments for your nodes, one that exposes the SPARK_WORKER_WEBUI_PORT
on 8081
and another on 8082
and also make sure that only one pod is scheduled per node. 如果每个节点有两个SPARK_WORKER_WEBUI_PORT
以为节点创建两个部署,一个部署在8081
上公开SPARK_WORKER_WEBUI_PORT
,另一个在8082
上公开,并确保每个节点仅调度一个Pod。
You can pass these values in the container environment varialbles 您可以在容器环境变量中传递这些值
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.