Where are Flink Job Manager and Task Manager hosted locally?

Question

I have Flink server running locally on port 8081. And I deploy jar of my flink application on it, which runs successfully.

Now I need to listen to Flink metrics from prometheus, for which I need to configure Job manager and Task manager as targets in prometheus.yml.

So where do I get ports for same?

Answer 1

configure the file /etc/prometheus/prometheus.yml only on the host that you JM is placed and point the IPs of the TM as targets. In my example the JM is at localhost and the TM are in localhost , 192.168.56.101 , 192.168.56.102 , and 192.168.56.103 .

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9090']
  - job_name: 'flink'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9250', 'localhost:9251', '192.168.56.101:9250', '192.168.56.102:9250', '192.168.56.103:9250']
    metrics_path: /

you have to configure the file conf/flink-conf.yaml to export data to Prometheus on a range of ports. This link explains in detail.

# Metrics Reporter
metrics.reporter.prom.class: org.apache.flink.metrics.prometheus.PrometheusReporter
metrics.reporter.prom.host: 127.0.0.1
metrics.reporter.prom.port: 9250-9260