Prometheus alertmanager persistent storage with docker

Question

For some reason I can't get persistent storage to work with alertmanager. This is my compose:

  alertmanager:
    image: 'my/alertmanager/prod:latest'
    restart: always
    volumes:
      - alertmanager-data:/alertmanager-data
    command:
      - '--config.file=/alertmanager/alertmanager-config.yml'
      - '--storage.path=/alertmanager-data'
    ports:
      - 9103:9093

  volumes:
    alertmanager-data:
      external: true

I have created the volume and everything seems to initialise correctly. But after entering the machine /alertmanager-data doesn't seem to contain any DB or files. And creating new silences on the alertmanager never persist.

Answer 1

Moved from comment for visibility

It turns out that it actually does work, but it takes a while for it to write memory to storage. I don't remember how often it is stored but it takes some time.

Answer 2

Looking at this change https://github.com/prometheus/alertmanager/pull/2849/files It looks like it used to be persisted to disk every 15 minutes, and now that is configurable with the parameter --data.maintenance-interval since version 0.25