Apache Flink Dashboard not showing metrics

Question

I have the following very simple Apache Flink Pipeline for which I would like to get some metrics, as explained in the Apache Flink documentation , via the Apache Flink Dashboard:

import org.apache.flink.api.common.functions.RichMapFunction;
import org.apache.flink.configuration.Configuration;
import org.apache.flink.metrics.Counter;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.source.RichSourceFunction;

public class Pipeline {

    public static void main(String[] args) throws Exception {
        final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

        env.addSource(new RichSourceFunction<String>() {
            private static final long serialVersionUID = 3990963645875188158L;
            private boolean notCanceled = true;

            @Override
            public void run(SourceContext<String> ctx) throws Exception {
                while (notCanceled) {
                    ctx.collect("test");
                }
            }

            @Override
            public void cancel() {
                notCanceled = false;
            }
        }).map(new RichMapFunction<String, String>() {
            private static final long serialVersionUID = 1L;
            private transient Counter counter;

            @Override
            public void open(Configuration parameters) throws Exception {
                super.open(parameters);
                this.counter = getRuntimeContext()
                        .getMetricGroup()
                        .counter("myCounter");
            }

            @Override
            public String map(String value) throws Exception {
                this.counter.inc();
                return "mappedtext";
            }
        }).print();

        env.execute();
    }

}

I do run that Pipeline using the Docker Setup available via Docker-Hub . Everything uses Apache Flink 1.10.0. The Pipeline runs fine, but when I try to view my metric I only get:

Now I ask myself what am I doing wrong? Is there some configuration parameter I am missing or is this not the correct place to view that metric in the dashboard? Can someone please advise or at least point me to a resource, where I would get more information?

Answer 1

I believe what's going on is that you have added an operator metric to the map operator, but the web ui is displaying task metrics. (In the case of this simple, embarrassingly parallel job, the source, map, and sink operators have been chained together into a single task.)

To inspect this metric you've added, you could use the REST API, or any of the metrics reporters. I think it may also show up in the web UI if you disable operator chaining via

    env.disableOperatorChaining();