Android CameraX 分析图像 stream

Question

I want to analyse a stream of image frames and do some computation on it. However, with CameraX, the ImageAnalysis implementation class seems to get called only once - on camera bind.

My question is: How do I run the analysis on a continuous stream of images - conceptually akin to a video stream?

Following is my camera, preview and analysis setup code:

private void setPreview() {
    ListenableFuture<ProcessCameraProvider> instance = ProcessCameraProvider.getInstance(this);

    Activity self = this;
    instance.addListener(() -> {

        try {
            ProcessCameraProvider cameraProvider = instance.get();
            Preview preview = new Preview.Builder().build();


            ImageAnalysis imageAnalysis = new ImageAnalysis.Builder().build();
            imageAnalysis.setAnalyzer(Executors.newFixedThreadPool(1),
                                                    new ImageAnalyser(new CameraLogListener() {
                                                        @Override
                                                        public void log(String msg) {
                                                            Log.e("Camera log", msg);
                                                        }
                                                    }));

            CameraSelector cameraSelector = new CameraSelector.Builder().requireLensFacing(CameraSelector.LENS_FACING_BACK).build();

            cameraProvider.unbindAll();
            Camera camera = cameraProvider.bindToLifecycle(((LifecycleOwner)self), cameraSelector, preview, imageAnalysis);

            preview.setSurfaceProvider(
                    ((PreviewView)findViewById(R.id.cameraTextureView)).createSurfaceProvider(camera.getCameraInfo()));

        } catch (ExecutionException e) {
            e.printStackTrace();
            Log.e("TAG", "Use case binding failed", e);
        } catch (InterruptedException e) {
            e.printStackTrace();
            Log.e("TAG", "Use case binding failed", e);
        }

    }, ContextCompat.getMainExecutor(this));

}

Following is my ImageAnalysis implementation class:

private class ImageAnalyser implements ImageAnalysis.Analyzer {

    CameraLogListener listener;
    public ImageAnalyser(CameraLogListener listener) {
        this.listener = listener;
    }

    @Override
    public void analyze(@NonNull ImageProxy image) {
        ByteBuffer imageBuffer = image.getPlanes()[0].getBuffer();
        StringBuilder sb = new StringBuilder();
        sb.append("format:" + image.getFormat()).append("\n")
            .append(image.getWidth() + " x " + image.getHeight()).append("\n\n");

        for (int i=0; i<image.getPlanes().length; i++) {
            sb.append("pixel stride:").append(image.getPlanes()[i].getPixelStride())
                .append("\nrow stride:").append(image.getPlanes()[i].getRowStride())
                .append("\n");
        }

        listener.log(sb.toString());

    }
}

Answer 1

I found the problem. At the end of the

  public void analyze(@NonNull ImageProxy image) {}

method, you'd need to call image.close(). Quoting from the documentation :

Before returning from analyze(), close the image reference by calling image.close() to avoid blocking the production of further images (causing the preview to stall) and to avoid potentially dropping images. The method must complete analysis or make a copy instead of passing the image reference beyond the analysis method.

Answer 2

Close the ImageProxy object, not the image, as stated on the Image analysis Doc

Release the ImageProxy to CameraX by calling ImageProxy.close(). Note that you shouldn't call the wrapped Media.Image's close function (Media.Image.close()).