Prometheus not expose all collected metrics

Question

I have configured Prometheus to scrape metrics for a few sources.

• cadvisor
• node-exporter
• prometheus
• 2 nodejs applications.
• hazelcast servers

The problem is.

In Prometheus UI I can see all targets and in graph I can query all metrics include nodejs related. But if I do curl http://localhost:9090/metrics I see only metrics related to prometheus and node-exporter. There is nothing related to nodejs applications or hazelcast.

Direct request to nodejs applications curl http://localhost:8080/metrics return all of those values without any problems.

What could cause such a problem?

Answer 1

This url : localhost:9090/metrics is only here to expose the Prometheus metrics. It is the one used by Prometheus to scrape itself. It has nothing to do with the application scraped by Prometheus.

Also, node related metrics that you see here, are not the one exposed by the node_exporter (there can be some similarities). If you want to see the node_exporter metrics, you need to go to localhost:9100/metrics (if you didn't change the default configuration).

Now to see the metrics scraped by Prometheus, you have the following options :

1. Use the Prometheus query explorer
2. Use another tool (Grafana for example)
3. Use the Prometheus API. Documentation here