简体繁体 English

Flink指标作为CSV？

[英]Flink metrics as csv?

原文 2018-01-24 17:38:34 3 1 performance/ apache-flink/ flink-streaming

I'm trying to test different metrics on some flink jobs. 我正在尝试测试某些flink作业的不同指标。 However, I found it pretty cumbersome to set up a metrics reporter. 但是，我发现设置指标报告器非常麻烦。 Currently I'm using Graphite and for each job I need to manually select all the metrics I want, put them into different graphs to get an overview whether this test is meaningful and if so, I have to export all metrics individually into a csv file and finally merge it again in another diagram (custom diagram is mandatory unfortunately). 当前，我正在使用Graphite，对于每项工作，我需要手动选择所需的所有指标，将它们放入不同的图形中以概述该测试是否有意义，如果是，则必须将所有指标分别导出到一个csv文件中最后将其再次合并到另一个图中（不幸的是，必须使用自定义图）。

Then I redeploy the job with different settings or parallism, which changes the job id, which means I need to put all graphs together again, export those metrics, and so on... 然后，我以不同的设置或并行度重新部署作业，这更改了作业ID，这意味着我需要再次将所有图形放在一起，导出那些指标，依此类推...

Is there a more comfortable way to get metrics of flink jobs as csv file? 有没有更舒适的方法来将flink作业的指标作为csv文件获取？ Would be nice to get the metrics of certain operators (those ids stay the same) somehow as csv, independent of the job id or taskmanager id. 无论工作ID或taskmanager ID如何，以某种方式获取某些操作员（这些ID保持不变）与csv的度量标准将非常好。

1 个解决方案

With the influxdb reporter it's pretty easy to setup scope formats that make it the metric naming really clean, and then you can do nice regular queries on the database to dump out groups of metrics. 使用influxdb报告程序，可以很容易地设置范围格式，使它的度量标准命名变得非常干净，然后您可以在数据库上执行常规查询，以转储度量标准组。 See https://github.com/jgrier/flink-stuff/tree/master/flink-influx-reporter . 参见https://github.com/jgrier/flink-stuff/tree/master/flink-influx-reporter 。