Metricbeat Prometheus Client Timeout using Pagespeed Exporter

Question

I am currently try to load data from Prometheus Pagespeed Exporter ( https://github.com/foomo/pagespeed_exporter ) directly into ELK using Metricbeat. It seems so, that the Call of Pagespeed Exporter requires more time than Metricbeats offers to scrape the required data. A Client Timeout occures:

unable to decode response from prometheus endpoint: error making http request: Get http://pagespeed-exporter-monitoring:9271/metrics: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

Currently the Request is cancelled after about 10s while the Timeout in metricbeat should be defined like Timeout = Period ( https://www.elastic.co/guide/en/beats/devguide/current/metricset-details.html ). The Period in my case is configured to 3600s.

metricbeat:
  deployment:
    metricbeatConfig:
      metricbeat.yml: |
        metricbeat.modules:
        - module: prometheus
          enabled: true
          period: 3600s
          hosts: ["pagespeed-exporter-monitoring:9271"]
          metrics_path: /metrics

Is there any option to increase the Timeout or is there any other issue?

Answer 1

You can override the timeout on the module config, https://www.elastic.co/guide/en/beats/metricbeat/current/configuration-metricbeat.html#_timeout

For example:

metricbeat:
  deployment:
    metricbeatConfig:
      metricbeat.yml: |
        metricbeat.modules:
        - module: prometheus
          enabled: true
          period: 3600s
          hosts: ["pagespeed-exporter-monitoring:9271"]
          metrics_path: /metrics
          timeout: 60s # adjust here