简体   繁体   English

向 PushGateway 报告 Flink 指标时,每个作业的作业名称不同

[英]Different jobName per Job when reporting Flink metrics to PushGateway

I'm using Flink 1.9.1 with PrometheusPushGateway to report my metrics.我使用 Flink 1.9.1 和 PrometheusPushGateway 来报告我的指标。 The jobName the metrics are reported with is defined in the flink-conf.yaml file which makes the jobName identical for all jobs who run on the cluster, but I want a different jobName to be reported for every running job.报告指标的作业名称在 flink-conf.yaml 文件中定义,这使得在集群上运行的所有作业的作业名称都相同,但我希望为每个正在运行的作业报告不同的作业名称。 To do so, I tried overriding the config's value inside the job before executing the Stream:为此,我尝试在执行 Stream 之前覆盖作业中的配置值:

Configuration conf = GlobalConfiguration.loadConfiguration();
    conf.setString(
            "metrics.reporter.promgateway.jobName",
            conf.getString("metrics.reporter.promgateway.jobName", "") + "-" + pipeline
    );
    final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    env.getConfig().setGlobalJobParameters(conf);

When pipeline is a String variable.pipeline是一个字符串变量时。

When running the job locally, it worked.在本地运行作业时,它起作用了。 But now I'm running flink in High Availability mode and it doesn't work anymore :( The config I override in the code is ignored and only the value in the cluster's flink-conf.yaml file is used.但是现在我在高可用性模式下运行 flink 并且它不再工作了:(我在代码中覆盖的配置被忽略,只使用集群的 flink-conf.yaml 文件中的值。

So how can I change the jobName per job?那么如何更改每个作业的作业名称? And if I can't, is there a way to set additional Labels when reporting the metrics?如果我不能,有没有办法在报告指标时设置额外的标签? Because I haven't seen an option for that as well.因为我还没有看到一个选项。

Thanks :)谢谢 :)

You can use the following steps to achieve this:您可以使用以下步骤来实现此目的:

  1. Pass jobName as a command argument like: --jobName MyJobNamejobName作为命令参数传递,例如:-- --jobName MyJobName

  2. Set global parameters:设置全局参数:

public static void main(String[] args) throws Exception {
    final ParameterTool command = ParameterTool.fromArgs(args);
    String jobName = command.getRequired("jobName");

    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    Configuration globalConfiguration = new Configuration();
    globalConfiguration.setString("jobName", jobName);
    env.getConfig().setGlobalJobParameters(globalConfiguration);

}
  1. Use it:用它:
ParameterTool parameters = (ParameterTool) getRuntimeContext().getExecutionConfig().getGlobalJobParameters();
parameters.getRequired("jobName");

Here is a link that may help you too: https://ci.apache.org/projects/flink/flink-docs-stable/dev/best_practices.html#parsing-command-line-arguments-and-passing-them-around-in-your-flink-application这是一个也可以帮助您的链接: https : //ci.apache.org/projects/flink/flink-docs-stable/dev/best_practices.html#parsing-command-line-arguments-and-passing-them-在你的 flink 应用程序中

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM