将框架保存到文件onFrameAvailable [Google Project Tango]

Question

I am trying to save to files the RGB frames along with Pose data, and then do some post processing on them. The main issue is that currently the only way to do this using the Tango Java API is to render to a GLSurfaceView , by connecting them via

tangoCameraPreview.connectToTangoCamera(mTango,TangoCameraIntrinsics.TANGO_CAMERA_COLOR);

Then you would use GlReadPixels to read the pixels into an array and save that to a file.

The problem with this is that GlReadPixels is slow . In fact, using this I am getting about 3-4 fps using what I just described.

Looking at other, more general answers on taking a photo bursts, I have seen various people saying that when using SurfaceView instead of GlSurfaceView , they managed to get up to 15fps.

I didn't find any way to use SurfaceView with the tango camera, since connectToTangoCamera needs a GlSurfaceView and I can't just use Camera and bind that to a SurfaceView because when I'm trying to open it (via Camera.open() ), it's already being used by Tango . Tango needs to use it in order to get the colorToIMUPose data.

So I'm really not sure what workaround might I find in order to be able to get at least 10fps.

Answer 1

You could use the C++ API with TangoService_connectOnFrameAvailable where you will get a YUV frame buffer in a reasonable speed. Checkout the tango c example video-overlay-jni-example where they do the RGB conversion. I use this way to interface OpenCV filters to the rendering process.