Golang Prometheus Exporter - Pull metrics on demand

Question

As defined in the prometheus documentation , when writing exporters it states the following:

Metrics should only be pulled from the application when Prometheus scrapes them, exporters should not perform scrapes based on their own timers.

The following code technically works fine, and publishes an appropriate page with my custom metric. So it solves my business need as-is.

However, it is inefficient and running an infinite loop to constantly update the value. This does not match the above mentioned practice from the documentation of only generating metric values when Prometheus scrapes.

package main

import (
    "log"
    "net/http"
    "time"

    "github.com/prometheus/client_golang/prometheus"
    "github.com/prometheus/client_golang/prometheus/promhttp"
)

func main() {
    var metricSystemTime = prometheus.NewGauge(prometheus.GaugeOpts{
        Name: "custom_system_time", Help: "Timestamp in unix epoch format"})

    prometheus.MustRegister(metricSystemTime)

    go startServer()

    for {
        metricSystemTime.Set(float64(time.Now().Unix()))
        time.Sleep(1 * time.Second)
    }
}

func startServer() {
    http.Handle("/metrics", promhttp.Handler())
    log.Fatal(http.ListenAndServe(":19100", nil))
}

In order to only pull metrics when prometheus scrapes the exporter endpoint, that means I need to somehow listen for GET requests on the http.ListenAndServe() object? How can I do that?

In my reading I found http.HandlerFunc() instead of http.Handler() which seems to be going down the right path, but I can't get it to work with promhttp.Handler() and I'm getting pretty lost on it.

Answer 1

The documentation you quoted is about writing an exporter. The exporter usually is an utility that extracts metrics from another application in a custom format and exposes them in prometheus format.

The code you showed is a correct way to instrument your own application. Your business logic updates the metric as it goes and the HTTP server serves the current metric value when requested.

Answer 2

Figured it out after doing a lot of searching. Hopefully this helps someone in the future.

Here is a working exporter that fits the requirement of only gathering metrics when a prometheus scrape happens

• No endless loops, no internal timers.
• Publishes new metrics every time Prometheus scrapes the exporter, as defined in the documentation from the original question.

package main

import (
    "log"
    "net/http"
    "time"

    "github.com/prometheus/client_golang/prometheus"
    "github.com/prometheus/client_golang/prometheus/promhttp"
)

type Collector struct {
    // Add one of the below lines for each collector you define in the newCollector() function
    systemTime    *prometheus.Desc
}

// Declare your exporter metrics here. Referred to as "collectors"
func newCollector() *Collector {
    return &Collector{
        systemTime: prometheus.NewDesc("my_system_time",
            "System timestamp in unix epoch format",
            nil, nil,
        ),
        // Add more collectors (aka metric definitions) here 
    }
}

func (collector *Collector) Describe(ch chan<- *prometheus.Desc) {
    // Add one of these lines for each of your collectors declared above
    ch <- collector.systemTime
}

// This fuction runs when Prometheus scrapes the exporter. It will set a new value for the metric(s)
// I have no idea how it works, but it does.
func (collector *Collector) Collect(ch chan<- prometheus.Metric) {

    var setSystemTime float64

    // Calculate the value of the metric
    setSystemTime = float64(time.Now().Unix())

    // Here is where you set the "new" value for the metric
    // This example is a Gauge, but it can be any Prom metric type
    ch <- prometheus.MustNewConstMetric(collector.systemTime, prometheus.GaugeValue, setSystemTime)
}

// Main function, spawns the collector and the web server
func main() {
    collector := newCollector()
    prometheus.MustRegister(collector)

    http.Handle("/metrics", promhttp.Handler())
    log.Fatal(http.ListenAndServe(":19100", nil))
}