[英]Flink: all traffic goes to one Task Manager (in cluster with 1 Job Manager + 2 Task Managers)
I have following set up:我有以下设置:
Installation type k8s: 1.18
Flink version: 1.12
1 Job Manager
2 Task Manager
In flink-conf.yaml of task manager在任务管理器的 flink-conf.yaml
flink-conf.yaml: |
state.backend: rocksdb
blob.server.port: 6124
jobmanager.rpc.port: 6123
parallelism.default: 2
queryable-state.proxy.ports: 6125
taskmanager.numberOfTaskSlots: 2
taskmanager.rpc.port: 6122
jobmanager.memory.process.size: 2900m
taskmanager.memory.process.size: 2900m
jobmanager.web.address: 0.0.0.0
rest.address: 0.0.0.0
rest.bind-address: 0.0.0.0
In flink-conf.yaml of job manager在作业管理器的 flink-conf.yaml
flink-conf.yaml: |
state.backend: rocksdb
blob.server.port: 6124
jobmanager.rpc.port: 6123
parallelism.default: 2
queryable-state.proxy.ports: 6125
taskmanager.rpc.port: 6122
jobmanager.memory.process.size: 2900m
taskmanager.memory.process.size: 2900m
jobmanager.web.address: 0.0.0.0
rest.address: 0.0.0.0
rest.bind-address: 0.0.0.0
rest.port: 8081
With above configuration, only one task manager is active ie gets traffic and another task manager remains idle, even though number of events increases to extreme level.使用上述配置,即使事件数量增加到极端水平,只有一个任务管理器处于活动状态,即获得流量,而另一个任务管理器保持空闲。
Please suggest, if I am missing anything?请建议,如果我遗漏了什么?
You have set the parallelism to 2, and given each task manager 2 slots.您已将并行度设置为 2,并为每个任务管理器分配了 2 个插槽。 Thus a single task manager can provide the requested parallelism, and that's what will happen by default.因此,单个任务管理器可以提供请求的并行性,这就是默认情况下会发生的事情。
If you want the scheduler to behave differently, you could set如果您希望调度程序表现不同,您可以设置
cluster.evenly-spread-out-slots: true
or you could reduce taskmanager.numberOfTaskSlots
to 1.或者您可以将taskmanager.numberOfTaskSlots
减少到 1。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.