如何從 Matlab 獲取的深度圖像和彩色圖像生成 3D 點雲

Question

我有 2 組數據從 Kinect 獲取 1- 深度圖像，大小為 480 640 (uint16) 來自場景 2- 彩色圖像，大小相同（480 640 * 3 單個）來自同一場景。

如何將這些數據合並在一起以在 Matlab 中生成具有 PLY 格式的彩色 3D 點雲？

不幸的是，我無法再訪問 Kinect，我應該只使用這些數據。

Answer 1

我從來沒有試過在matlab中這樣做，但我認為這就是你要找的：

http://es.mathworks.com/help/vision/ref/pcfromkinect.html

該工具位於Computer Vision System Toolbox™內。

Answer 2

好問題。 您應該使用Burrus的本教程 - 基本上您需要使用深度信息將顏色/深度質心轉換為第三維。 請注意，Kinect v1深度和顏色流稍有不匹配，因此也要考慮到這一點。

教程可以在這里找到： http ：//nicolas.burrus.name/index.php/Research/KinectCalibration您也可以使用這個作者的作品：Khoshelham，K。，＆Elberink，SO（2012） - 准確性和分辨率用於室內地圖繪制應用的Kinect深度數據

matlab代碼應該是這樣的：

% All formulas and values from:
% Khoshelham, K., & Elberink, S. O. (2012).
% Accuracy and resolution of Kinect depth data for indoor mapping applications.
% Sensors (Basel, Switzerland), 12(2), 1437–54. doi:10.3390/s120201437

load('janFrameThousand.mat')
pc=zeros([size(D) 3]);
W=size(D,2);
H=size(D,1);
f=5.453;
for indWidth = 1:W
    for indHeight= 1:H
        % copy z value
        pc(indHeight,indWidth,3)=D(indHeight,indWidth);
        % calc x value
        pc(indHeight,indWidth,1)=-(pc(indHeight,indWidth,3)/f)*...
            ((indWidth-W/2)*0.0093+0.063);
        % calc y value
        pc(indHeight,indWidth,2)=-(pc(indHeight,indWidth,3)/f)*...
            ((indHeight-H/2)*0.0093+0.039);
    end
end
X=pc(:,:,1);
% X=X(:);
Y=pc(:,:,2);
% Y=Y(:);
Z=-pc(:,:,3);
Z(Z==0)=NaN;

Surface=surf(X,Y,Z,'edgecolor','none','facecolor','interp');
lighting gouraud
camlight
% colormap(repmat(winter,20,1))
axis image
axis vis3d
xlabel('X axis')
ylabel('Y axis')
zlabel('Z axis')

Answer 3

這是將 RGB-D 圖像轉換為點雲的 MATLAB OOP 代碼，提供相機的內在參數：

% Class definition
classdef rgbd2pointcloud < handle
    
    % Public properties
    properties
        color
        depth
        json_filename
        cam_intrinsics
    end
    % Public methods
    methods
        % Constructor
        function obj = rgbd2pointcloud(color, depth, cam_intrinsics, json_filename)
            if isstring(color)
                obj.color = imread(color);
            else
                obj.color = color;
            end
            if isstring(depth)
                obj.depth = double(imread(depth));
            else
                obj.depth = double(depth);
            end
            obj.json_filename = json_filename;
            obj.cam_intrinsics = cam_intrinsics;
        end
        % Get the XYZ and RGB data
        function [xyz, rgb] = xyz_rgb(obj)
            cam_params = get_camera_intrinsics(obj);
    
            xx = 1 : cam_params.width;
            yy = 1 : cam_params.height;
            [xv, yv] = meshgrid(xx, yy);
                  
            u = reshape(xv', 1, []);
            v = reshape(yv', 1, []);
    
            z = reshape(obj.depth', 1, []) / cam_params.depth_scale;
            x = (u - cam_params.cx) .* z / cam_params.fx;
            y = (v - cam_params.cy) .* z / cam_params.fy;
            
            xyz = [x; y; z]';
            rgb = reshape(pagetranspose(obj.color), [], 3);
        end 
        % Write to the point cloud
        function write2file(obj, xyz, rgb, file_name)
            ptCloud = pointCloud(xyz, Color = rgb);
            pcwrite(ptCloud,file_name, 'Encoding', 'compressed')
        end
    end
    % Private methods
    methods (Access = private)
        % JSON reader
        function cam_params = readJSON(obj)
            fid = fopen(obj.json_filename); 
            raw = fread(fid,inf); 
            str = char(raw'); 
            fclose(fid); 
            
            cam_params = jsondecode(str);
        end
        % Get the camera intrinsics
        function cam_params = get_camera_intrinsics(obj)
            if ~isempty(obj.json_filename)
                cam_params = readJSON;
            else
                cam_params = obj.cam_intrinsics;
            end
        end        
    end
end

下面是代碼使用：

%% Inputs
% Read the RGB and D images
color = imread("redwood_847.png");
depth = imread("redwood_847d.png");

% Json filename
json_filename = [];

% Output point cloud filename
file_name = 'output.pcd';

% Camera intrinsics
camera_intrinsic.cx              = 319.5;
camera_intrinsic.cy              = 239.5;
camera_intrinsic.fx              = 525;
camera_intrinsic.fy              = 525;
camera_intrinsic.depth_scale     = 1000; % Depth scale (constant) to convert mm to m vice-versa
camera_intrinsic.width           = 640;
camera_intrinsic.height          = 480;

%% Object callback    
obj = rgbd2pointcloud(color, depth, camera_intrinsic, json_filename);
[xyz, rgb] = obj.xyz_rgb();
obj.write2file(xyz, rgb, file_name)

%% Display the point cloud
figure;
pcshow('output.pcd', 'VerticalAxis', 'x', 'VerticalAxisDir', 'down')
xlabel("X")
ylabel("Y")
zlabel("Z")

我也已將此代碼上傳到MATLAB 文件交換