RGBIR的Tango相机预览

Question

我正在使用Tango的videoOverlaySample演示。

我希望看到的不是颜色，而是红外数据（单独或彩色）。 因此，我在出现的两个地方都将TANGO_CAMERA_COLOR替换为TANGO_CAMERA_RGBIR。

但是屏幕是黑色的。

这是代码：

/*
 * Copyright 2014 Google Inc. All Rights Reserved.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package com.projecttango.experiments.videooverlaysample;

import java.util.ArrayList;

import android.app.Activity;
import android.content.Intent;
import android.os.Bundle;
import android.util.Log;
import android.widget.Toast;

import com.google.atap.tangoservice.Tango;
import com.google.atap.tangoservice.Tango.OnTangoUpdateListener;
import com.google.atap.tangoservice.TangoCameraIntrinsics;
import com.google.atap.tangoservice.TangoCameraPreview;
import com.google.atap.tangoservice.TangoConfig;
import com.google.atap.tangoservice.TangoCoordinateFramePair;
import com.google.atap.tangoservice.TangoEvent;
import com.google.atap.tangoservice.TangoPoseData;
import com.google.atap.tangoservice.TangoXyzIjData;

/**
 * An example showing the usage of TangoCameraPreview class
 * Usage of TangoCameraPreviewClass:
 * To use this class, we first need initialize the TangoCameraPreview class with the activity's 
 * context and connect to the camera we want by using connectToTangoCamera class.Once the connection 
 * is established we need to manually update the TangoCameraPreview's texture by using the
 * onFrameAvailable callbacks.
 * Note:
 * To use TangoCameraPreview class we need to ask the user permissions for MotionTracking 
 * at the minimum level. This is because in Java all the call backs such as 
 * onPoseAvailable,onXyzIjAvailable, onTangoEvents, onFrameAvailable are set together at once. 
 */
public class MainActivity extends Activity {
    private TangoCameraPreview tangoCameraPreview;
    private Tango mTango;
    private boolean mIsConnected;
    private boolean mIsPermissionGranted;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        tangoCameraPreview = new TangoCameraPreview(this);
        mTango = new Tango(this);
        startActivityForResult(
                Tango.getRequestPermissionIntent(Tango.PERMISSIONTYPE_MOTION_TRACKING),
                Tango.TANGO_INTENT_ACTIVITYCODE);
        setContentView(tangoCameraPreview);
    }

    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        // Check which request we're responding to
        if (requestCode == Tango.TANGO_INTENT_ACTIVITYCODE) {
            // Make sure the request was successful
            if (resultCode == RESULT_CANCELED) {
                Toast.makeText(this, "Motion Tracking Permissions Required!",
                        Toast.LENGTH_SHORT).show();
                finish();
            } else {
                startCameraPreview();
                mIsPermissionGranted = true;
            }
        }
    }

    // Camera Preview
    private void startCameraPreview() {
        // Connect to color camera
        tangoCameraPreview.connectToTangoCamera(mTango,
                TangoCameraIntrinsics.TANGO_CAMERA_RGBIR);
        // Use default configuration for Tango Service.
        TangoConfig config = mTango.getConfig(TangoConfig.CONFIG_TYPE_DEFAULT);
        config.putBoolean(TangoConfig.KEY_BOOLEAN_DEPTH, true);
        mTango.connect(config);
        mIsConnected = true;

        // No need to add any coordinate frame pairs since we are not using 
        // pose data. So just initialize.
        ArrayList<TangoCoordinateFramePair> framePairs = new ArrayList<TangoCoordinateFramePair>();
        mTango.connectListener(framePairs, new OnTangoUpdateListener() {
            @Override
            public void onPoseAvailable(TangoPoseData pose) {
                // We are not using OnPoseAvailable for this app
            }

            @Override
            public void onFrameAvailable(int cameraId) {

                // Check if the frame available is for the camera we want and
                // update its frame on the camera preview.
                if (cameraId == TangoCameraIntrinsics.TANGO_CAMERA_RGBIR) {
                    tangoCameraPreview.onFrameAvailable();
                }
            }

            @Override
            public void onXyzIjAvailable(TangoXyzIjData xyzIj) {
                // We are not using OnPoseAvailable for this app
            }

            @Override
            public void onTangoEvent(TangoEvent event) {
                // We are not using OnPoseAvailable for this app
            }
        });
    }

    @Override
    protected void onPause() {
        super.onPause();
        if(mIsConnected) {
            mTango.disconnect();
            tangoCameraPreview.disconnectFromTangoCamera();
            mIsConnected = false;
        }
    }

    @Override
    protected void onResume() {
        super.onResume();
        if (!mIsConnected && mIsPermissionGranted) {
            startCameraPreview();
        }
    }
}

Answer 1

你不能用

tangoCameraPreview.connectToTangoCamera(mTango,TangoCameraIntrinsics.TANGO_CAMERA_RGBIR);

Java API不提供具有RGBIR的connectToTangoCamera，仅提供彩色和鱼眼摄像头（请参见此处 ）

要显示深度图像而不是彩色图像，必须手动计算深度图像。 因此，您大致必须：

1. 将具有给定时间戳记的姿态的点云转换为相机帧。
2. 投影点云点（x，y，z）以接收像素（x，y）
3. 您需要一个具有摄像机框架分辨率（1240x720）的阵列。
4. 用0（黑色）填充数组。
5. 将z值存储在给定的像素位置（对像素进行升采样，因为深度相机的分辨率仅为320x180）
6. 在OpenGL中，您可以轻松地将该阵列用作纹理。

有关更多详细信息，建议您查看C示例rgb-depth-sync