Prometheus 无法启动：无法使用快照程序 btrfs

Question

I have prometheus installed on an Amazon linux instance and here is the status of the service:

[ec2-user@ip-10-193-192-56 ~]$ sudo systemctl status prometheus
● prometheus.service - Prometheus Server
   Loaded: loaded (/etc/systemd/system/prometheus.service; enabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Tue 2020-01-28 16:27:41 UTC; 16h ago
     Docs: https://prometheus.io/docs/introduction/overview/
 Main PID: 21129 (code=exited, status=1/FAILURE)

Jan 28 16:27:40 ip-10-193-192-56.service.essilor systemd[1]: prometheus.service: main process exited, code=exited, status=1/FAILURE
Jan 28 16:27:40 ip-10-193-192-56.service.essilor systemd[1]: Unit prometheus.service entered failed state.
Jan 28 16:27:40 ip-10-193-192-56.service.essilor systemd[1]: prometheus.service failed.
Jan 28 16:27:41 ip-10-193-192-56.service.essilor systemd[1]: prometheus.service holdoff time over, scheduling restart.
Jan 28 16:27:41 ip-10-193-192-56.service.essilor systemd[1]: start request repeated too quickly for prometheus.service
Jan 28 16:27:41 ip-10-193-192-56.service.essilor systemd[1]: Failed to start Prometheus Server.
Jan 28 16:27:41 ip-10-193-192-56.service.essilor systemd[1]: Unit prometheus.service entered failed state.
Jan 28 16:27:41 ip-10-193-192-56.service.essilor systemd[1]: prometheus.service failed.
[ec2-user@ip-10-193-192-56 ~]$

When I show the logs I get the following:

[ec2-user@ip-10-193-192-56 ~]$ sudo cat /var/log/messages | grep "error"
Jan 28 16:27:37 ip-10-193-192-56 prometheus: level=error ts=2020-01-28T16:27:37.650Z caller=main.go:736 err="opening storage failed: block dir: \"/app/prometheus/data/01DZ9119BY4ZGCSRF1H27TDXSB\": open /app/prometheus/data/01DZ9119BY4ZGCSRF1H27TDXSB/meta.json: no such file or directory"
Jan 28 16:27:38 ip-10-193-192-56 prometheus: level=error ts=2020-01-28T16:27:38.716Z caller=main.go:736 err="opening storage failed: block dir: \"/app/prometheus/data/01DZ9119BY4ZGCSRF1H27TDXSB\": open /app/prometheus/data/01DZ9119BY4ZGCSRF1H27TDXSB/meta.json: no such file or directory"
Jan 28 16:27:39 ip-10-193-192-56 prometheus: level=error ts=2020-01-28T16:27:39.340Z caller=main.go:736 err="opening storage failed: block dir: \"/app/prometheus/data/01DZ9119BY4ZGCSRF1H27TDXSB\": open /app/prometheus/data/01DZ9119BY4ZGCSRF1H27TDXSB/meta.json: no such file or directory"
Jan 28 16:27:40 ip-10-193-192-56 prometheus: level=error ts=2020-01-28T16:27:40.142Z caller=main.go:736 err="opening storage failed: block dir: \"/app/prometheus/data/01DZ9119BY4ZGCSRF1H27TDXSB\": open /app/prometheus/data/01DZ9119BY4ZGCSRF1H27TDXSB/meta.json: no such file or directory"
Jan 28 16:27:40 ip-10-193-192-56 prometheus: level=error ts=2020-01-28T16:27:40.946Z caller=main.go:736 err="opening storage failed: block dir: \"/app/prometheus/data/01DZ9119BY4ZGCSRF1H27TDXSB\": open /app/prometheus/data/01DZ9119BY4ZGCSRF1H27TDXSB/meta.json: no such file or directory"
Jan 28 16:27:46 ip-10-193-192-56 containerd: time="2020-01-28T16:27:46.134041576Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
Jan 28 16:27:46 ip-10-193-192-56 containerd: time="2020-01-28T16:27:46.136910879Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.aufs" error="modprobe aufs failed: "modprobe: FATAL: Module aufs not found in directory /lib/modules/4.14.77-81.59.amzn2.x86_64\n": exit status 1"
Jan 28 16:27:46 ip-10-193-192-56 containerd: time="2020-01-28T16:27:46.138682614Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.zfs" error="path /var/lib/containerd/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter"
Jan 28 16:27:46 ip-10-193-192-56 containerd: time="2020-01-28T16:27:46.139296419Z" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/containerd/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter"
Jan 28 16:27:46 ip-10-193-192-56 containerd: time="2020-01-28T16:27:46.139604765Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter"
Jan 28 16:27:46 ip-10-193-192-56 containerd: time="2020-01-28T16:27:46.139907894Z" level=warning msg="could not use snapshotter aufs in metadata plugin" error="modprobe aufs failed: "modprobe: FATAL: Module aufs not found in directory /lib/modules/4.14.77-81.59.amzn2.x86_64\n": exit status 1"
Jan 28 16:27:46 ip-10-193-192-56 containerd: time="2020-01-28T16:27:46.159112796Z" level=error msg="Failed to load cni during init, please check CRI plugin status before setting up network for pods" error="cni config load failed: no network config found in /etc/cni/net.d: cni plugin not initialized: failed to load cni config"
[ec2-user@ip-10-193-192-56 ~]$

I have many errors here and I don't know what to do much about them. I tried to delete the broken block /app/prometheus/data/01DZ9119BY4ZGCSRF1H27TDXSB but it didn't change anything.

/app/prometheus/data is in an AWS EFS.

Answer 1

删除目录/app/prometheus/data/01DZ9119BY4ZGCSRF1H27TDXSB并重启服务器。