What's relationship between preview image size and captured image size in Android camera2 API?

Question

I am using camera2 API to building an image processing project in Android. I use TextureView to get a preview screen, and ImageReader to obtain image frames. For the purpose of image processing, I need to obtain frames with maximal size in ImageReader, and don't has any requirements on the size of preview frames. So I set the size of frames respectively with following codes:

        StreamConfigurationMap map = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
        imageDimension = map.getOutputSizes(SurfaceTexture.class)[0];
        texture.setDefaultBufferSize(imageDimension.getWidth(), imageDimension.getHeight());

        List<Size> outputSizes = Arrays.asList(map.getOutputSizes(sImageFormat));
        largest = Collections.max(outputSizes, new CompareSizesByArea());
        mImageReader = ImageReader.newInstance(largest.getWidth(), largest.getHeight(), sImageFormat, 1);

In may testing Android device Huawei P9: imageDimension is:1280*720; largest is: 3968*2976.

I found that the preview screen works well, however, the captured image in the onImageAvailable is completely black, I guess that nothing is captured.

I tried two ways to figure out what's going on here. 1) I set the size of ImageReader to imageDimension and found that the captured image is OK. 2) I set the size of preview frames to largest and found the preview screen is black.

Hence, I wonder if the size of captured image in ImageReader should less than that of preview frames. It it is the case, how can I set the maximal resolution of preview frame?

Anyway, my purpose is to obtain frames with maximal resolution as well as fluent preview screen. Any suggestions are welcome. Thanks.

Answer 1

In theory, the resolution of two separate outputs should make no difference, as long as you're within the resolution guarantees for multiple streams listed in the tables at the documentation for CameraDevice.createCaptureSession() . If you're operating at resolution above those guarantees, things may not work as expected.

In general, I'd recommend keeping the preview resolution <= 1080p (1920x1080); many devices cannot produce smooth 30fps preview at resolutions above that size; some may even require 720p or lower. Also, the GPU on a device may not actually be able to handle full-resolution camera output, which may explain some of the black preview output.

There's no reason that the ImageReader needs to be smaller than the preview; I suspect you may just have too high of a preview size.

So try a configuration of:

• Full-resolution ImageReader
• 1080p or similar preview

Answer 2

在某些设备上，如果捕获的纵横比不适合预览的纵横比，则捕获可能需要更长的时间。