Exact depth distance from Realsense D435 with X,Y coordinates

Question

I would like to calculate the depth distance with the x,y coordinates of a detect object.

In the image below, I used background subtraction in opencv to detect and track a new object that has entered in the cameras view. I am able to get the x,y coordinate with relative ease but have trouble getting the z depth with the realsense sdk. Is it possible in any way for me to input the x,y coordinates into the realsense sdk and get the z depth from there?

I'm using opencv python and realsense sdk 2 for reference.

Getting the z depth in the midpoint of the bounding box

import numpy as np
import cv2 as cv
import pyrealsense2 as rs

# Create a pipeline
pipeline = rs.pipeline()
config = rs.config()
config.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, 30)
config.enable_stream(rs.stream.color, 1280, 720, rs.format.bgr8, 30)

#Start pipeline
profile = pipeline.start(config)

erodeKernel = cv.getStructuringElement(cv.MORPH_RECT, (5,5))

fgbg = cv.createBackgroundSubtractorMOG2()
while True:
    frames = pipeline.wait_for_frames()
    depth_frame = frames.get_depth_frame()
    colour_frame = frames.get_color_frame()

    color_image = np.asanyarray(colour_frame.get_data())
    depth_image = np.asanyarray(depth_frame.get_data())

    # Apply colormap on depth image (image must be converted to 8-bit per pixel first)
    depth_colormap = cv.applyColorMap(cv.convertScaleAbs(depth_image, alpha=0.03), cv.COLORMAP_JET)

    blur = cv.GaussianBlur(color_image,(5,5),0)
    fgmask = fgbg.apply(blur)

    im2, contours, hierarchy = cv.findContours(fgmask.copy(), cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)
    for c in contours:
        if cv.contourArea(c) < 200:
            continue

        (x,y,w,h) = cv.boundingRect(c)
        cv.rectangle(color_image, (x,y), (x + w, y + h), (0,255,0), 2)

    cv.imshow('RealSense', color_image)
    cv.imshow("Depth", depth_colormap)
    cv.imshow('Mask', fgmask)
    if cv.waitKey(25) == ord('q'):
        break
cv.destroyAllWindows()
pipeline.stop()

Answer 1

It appears this is a rather simple solution. After poking around the c++ examples, the realsense sdk provides a function known as get_distance(x,y) where it will return the depth distance according to the x,y coords.

Note that this function in python is exactly the same but must be called from the depth frame and the x and y must be cast to integer

pipeline = rs.pipeline()
config = rs.config()
profile = pipeline.start(config)
frames = pipeline.wait_for_frames()
    while True:
        depth_frame = frames.get_depth_frame()
        zDepth = depth_frame.get_distance(int(x),int(y))