how to get disk-usage of ceph by python

Question

I want to get the disk-usage of my volumes that is on ceph.

rbd -p volumes list
volume-3968ca3d-d55f-47ac-9174-ff2614b0bde1
volume-493eee5c-3cdb-4c1c-b8d0-a24efba0c884

now I can get the usage by its ID

rbd disk-usage volumes/volume-3968ca3d-d55f-47ac-9174-ff2614b0bde1
NAME                                               PROVISIONED USED
volume-3968ca3d-d55f-47ac-9174-ff2614b0bde1@yasin1       1024M 532M

everything work fine. I want to do this command by API . I read https://docs.ceph.com/docs/mimic/rados/api/python/ hundreds of time and can not find any solution to get the usage by python.

what can I do to get usage by pyhton?

Answer 1

Everything is in the Rados and RBD documentation.

Your code will look something like the following snippet. It will connect to the cluster. It will get its IO context, create an RBD instance which in turn can retrieve image_names and the image_name is used to access the image instances that eventually have the size() method. This will give us the maximum size of the image. To calculate the disk usage the diff_iterate method is used to scan the whole image for existing/non-existing objects of the image. For every object the cb_offset method of the counter instance is called. The count is increased by the length of the object if the object exists. For images without fast diff map this takes a while. The same technique is used to diff snapshots. In this case the third parameter of diff_iterate is pointing to the snapshot. In our case we like to diff against the beginning of time (an empty image), so it is None.

import rados
import rbd

class DiffCounter:
    def __init__(self):
        self.count = 0
    def cb_offset(self, offset, length, exists):
        if exists:
            self.count+=length

def disk_usage():
    cluster = rados.Rados(conffile='/etc/ceph/ceph.conf')
    cluster.connect()
    try:
        ioctx = cluster.open_ioctx('rbd')
        try:
            rbd_inst = rbd.RBD()
            try:
                for image_name in rbd_inst.list(ioctx):
                    image = rbd.Image(ioctx, image_name)
                    max_size = image.size()
                    counter = DiffCounter()
                    image.diff_iterate(0,max_size,None,counter.cb_offset)
                    current_size = counter.count
                    print(image_name,max_size,current_size)
            finally:
                image.close()
        finally:
            ioctx.close()
    finally:
        cluster.shutdown()

if __name__ == '__main__':
    disk_usage()