简体繁体 English

Flink：在Flink集群上执行Jar文件

[英]Flink: Jar file execution on Flink cluster

原文 2017-02-23 07:52:17 6 1 java/ jar/ apache-flink/ flink-streaming

I am using the flink cluster with 9 task managers on 2 different physical nodes. 我将flink群集与2个不同物理节点上的9个任务管理器一起使用。 When i submit the jar to the Jobmanager(Master) of Flink, what does exactly happen to the jar. 当我将罐子提交给Flink的Jobmanager（Master）时，罐子到底发生了什么。 I mean, does the jar get distributed to all worker nodes or some part of the function gets reaplicated to all working nodes? 我的意思是，jar是否分布到所有工作节点，还是函数的某些部分重复到所有工作节点？

I am asking this because, when I trying to run my application, it is running locally successfully, but give an error of serialization on FlatMapFunction when subject to run on cluster. 我之所以这样问是因为，当我尝试运行我的应用程序时，它在本地成功运行，但是如果要在群集上运行，则会在FlatMapFunction上给出序列化错误。

1 个解决方案

Job Manager process application code, split it to tasks and subtasks and send them to one or more(depending on task parallelism) execution slots in task managers. 作业管理器处理应用程序代码，将其拆分为任务和子任务，然后将它们发送到任务管理器中的一个或多个（取决于任务并行性）执行插槽。 From the Flink documentation : 从Flink 文档中：

Each worker (TaskManager) is a JVM process, and may execute one or more subtasks in separate threads. 每个工作程序（TaskManager）是一个JVM进程，并且可以在单独的线程中执行一个或多个子任务。 To control how many tasks a worker accepts, a worker has so called task slots (at least one).] 为了控制一个工人接受多少个任务，一个工人有一个所谓的任务槽（至少一个）。

查看此图