16-bit depth image getting corrupted

Question

I am using realsense D435 and taking 16-bit depth data and converting into opencv 16-bit Mat and i am having custom-queue written in c++ where i am pushing all my frames and when i am popping out from queue the 16-bit data got corrupted.while pushing frame into queue i am saving into disk and while popping frame from queue that time also i am saving frame into disk.so from there i get to know frame got corrupted while popping. Same thing i tried with 8-bit data means while converting to opencv mat i converted to 8-bit Mat instead of 16-bit Mat and while popping 8-bit data.The data not get corrupted.

Code:

rs2::frameset data = pipe.wait_for_frames(); 
data = align.process(data);         
rs2::frame depth = data.get_depth_frame();
rs2::frame color = data.get_color_frame();

cv::Mat depth_image;        
cv::Mat fin_depth_image(cv::Size(WIDTH,HEIGHT), CV_16UC1, (void*)depth.get_data(), cv::Mat::AUTO_STEP);
cv::Mat color_image(cv::Size(WIDTH,HEIGHT), CV_8UC3, (void*)color.get_data(), cv::Mat::AUTO_STEP);
fin_depth_image.convertTo(depth_image,CV_8UC1);

cv::split(color_image,channels);        
channels.push_back(depth_image);
cv::merge(channels,fin_rgbd_img);           

camera_queue.push(std::make_pair(fin_rgbd_img,fin_depth_image));

Custom_queue Code:

inline void push(const T& elem) {
    {
        std::unique_lock<std::mutex> lock(_mutex);

        // wait for timeout while the queue is full
        _not_full.wait_for(lock,std::chrono::milliseconds(5));

        // If the queue is full, remove the old item and add the new item
        if (_queue.size() >= _capacity)
        {
            _queue.pop();
        }
        _queue.push(elem);
    }
    _not_empty.notify_all();
}

inline bool pop(T &elem) {
    bool status = false;
    {
        std::unique_lock<std::mutex> lock(_mutex);

        // wait for timeout while the queue is empty
        _not_empty.wait_for(lock,std::chrono::milliseconds(5));

        if (_queue.size() > 0)
        {
            elem = _queue.front();
            _queue.pop();
            status = true;
        }
    }
    _not_full.notify_one();
    return status;
}

So from above you can see that i am pushing fin_rgbd_img,fin_depth_image in queue. fin_rgbd_img is 8bit 4 channel Mat and fin_depth_image is 16-bit Mat. so while pushing i am saving also this 16-bit depth Mat by imwrite function that time i am having proper image. While popping out this 16-bit Mat got corrupted.

Below is the png image while pushing into queue i am saving to disk and it's proper and while popping complete black image filled with 0 is coming

Answer 1

Remember that both rs2::frame<\/code> and cv::Mat<\/code> are basically thin wrappers around smart pointers that manage the actual data. When you use the

Once the last copy of the rs2::frame depth<\/code> goes out of scope, the RealSense library frees the data block, and you have no control over what happens to it.

The other thing you can do (at the cost of more processing time) is use cv::Mat::clone()<\/code> and let OpenCV manage its own copy of the data, like this:

Instead use rs2::depth_frame<\/code> for depth images, and rs2::video_frame<\/code> for color or grayscale images. The reason is that using the base class makes it impossible to use features like rs2::video_frame::get_height()<\/code> & rs2::video_frame::get_width()<\/code> . The inheritance goes like this:

class frame ...
class video_frame : public frame ...
class depth_frame : public video_frame ...