如何在Flink中增加SinkFunction的numRecordsOutPut指标？

Question

I'm using flink to consume kafka and write to redis.

Here is my sink function to redis:

            .addSink(new RichSinkFunction<MobilePageEvent>() {

                @Override
                public void invoke(MobilePageEvent event, Context context) {

                    JEDIS_CLUSTER.zadd(..);
                }
            })
            .name("redis sink");

Though I can get the data from redis command line, but the metrics shows the output of sink function is zero:

How can I increment this metrics?

Answer 1

The numRecordsIn and numRecordsOut metrics count only the stream records that flow within the Flink job itself, and don't include communication to external systems. So in other words, sources don't report any records coming in, and sinks don't report any records going out.

As I see it you have a couple of options:

1. use the numRecordsIn metric on the sink as an approximation to what you want to know
2. fork or extend the RedisSink and add the metric you want

The pattern for adding a Counter metric is shown here .

In the case of the redis sink you could initialize a Counter in the open() method, and increment it in the invoke(). But that seems rather pointless, since this will simply mirror the numRecordsIn metric. If your redis sink is doing buffered bulk writes, then it might be more meaningful to wait to increment the metric until the data is actually sent to redis -- and in that case you might rather use a Meter than a Counter.