cadvisor machine_cpu_cores metrics comes every 5 minutes

Question

I am using Victoriametrics, cadvisor and docker to find our per container cpu usage. I am running the following query against cadvisor metrics

(sum(rate(container_cpu_usage_seconds_total{name != ''}[1m])) BY (instance, name) * 100) / ignoring(name) (sum by (instance) (machine_cpu_cores))

This should divide cpu_usage by machine_cpu_cores, but the metric machine_cpu_cores is available every 5 minutes, inspite of having a shorter collection interval in my config

global:
  scrape_interval: 10s

scrape_configs:
  - job_name: node_exporter
    static_configs:
      - targets: ["localhost:9100"]

  - job_name: cadvisor
    static_configs:
      - targets: ["localhost:8080"]

This leads to the following break in the graph

This metric should be available at at least a 1 minute resolution. Has anyone else faced these issues with cadvisor?

Answer 1

Could you please check what returns the following queries on the same time interval:

scrape_interval(container_cpu_usage_seconds_total[5m])
scrape_interval(machine_cpu_cores[5m])

Do you have any non-default flags configured for your VictoriaMetrics process?

Answer 2

I was able to achieve this by using the on (instance) group_left() expression instead of the ignoring flag

(sum(rate(container_cpu_usage_seconds_total{name,= ''}[1m])) BY (instance, name) * 100) / on(instance) group_left() (sum by (instance) (machine_cpu_cores))