How to explain Ceph space usage

Question

I looked up RBD disk space usage, but found different statistics from Ceph and the host which mounts the disk.

From Ceph:

$ rbd -p rbd du
NAME                                                        PROVISIONED    USED
kubernetes-dynamic-pvc-13a2d932-6be0-11e9-b53a-0a580a800339       40GiB 37.8GiB

From the host which mounts the disk

$ df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/rbd0                39.2G     26.6G     10.6G  72% /data

How could I explain the difference?

Answer 1

You can check mount options of /dev/rbd0 device. There should be no 'discard' option. Without that option filesystem cannot report to Ceph about reclaimed space. So Ceph has no idea how much space is actually occupied on rbd volume. This is not a big problem and can be safely ignored. You can rely on stats reported by kubelet.