ImageAnalysis 用例 CameraX 人脸检测

Question

I'm trying to develop an app that analyzes the live frames of the camera using CameraX and the MLKit Facedetecion API . It works fine in the preview use-case, but the image analysis use case doesn't work at all.

I don't understand why because I'm following the latest official documentation. My code:

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    previewView = findViewById(R.id.previewView);
    if (previewView == null) {
        Toast.makeText(this,
            "PreviewView not found",
            Toast.LENGTH_SHORT).show();
    }
    iv = findViewById(R.id.face_image_view);
    if (iv == null) {
        Toast.makeText(this,
            "ImageView not found",
            Toast.LENGTH_SHORT).show();
    }

    cameraProviderFuture = ProcessCameraProvider.getInstance(this);

    cameraProviderFuture.addListener(() - > {
        try {
            ProcessCameraProvider cameraProvider = cameraProviderFuture.get();
            if (allPermissionsGranted()) {
                bindPreview(cameraProvider);
                Log.i(TAG, "here1");
            } else {
                ActivityCompat.requestPermissions(this, REQUIRED_PERMISSIONS, REQUEST_CODE_PERMISSION);
            }


        } catch (ExecutionException | InterruptedException e) {
            // No errors need to be handled for this Future.
            // This should never be reached.
        }
    }, ContextCompat.getMainExecutor(this));
}

void bindPreview(@NonNull ProcessCameraProvider cameraProvider) {
    cameraProvider.unbindAll();

    //bind preview
    Preview preview = new Preview.Builder()
        .build();

    CameraSelector cameraSelector = new CameraSelector.Builder()
        .requireLensFacing(CameraSelector.LENS_FACING_FRONT)
        .build();

    preview.setSurfaceProvider(previewView.getSurfaceProvider());

    //bind image analysis
    ImageAnalysis imageAnalysis =
        new ImageAnalysis.Builder()
        .setTargetResolution(new Size(previewView.getWidth(), previewView.getHeight()))
        .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
        .build();

    imageAnalysis.setAnalyzer(executor, new ImageAnalysis.Analyzer() {
        @Override
        public void analyze(@NonNull ImageProxy imageProxy) {

            int rotationDegrees = imageProxy.getImageInfo().getRotationDegrees();

            Log.i(TAG, "here analyzer");
            @SuppressLint("UnsafeOptInUsageError") Image mediaImage = imageProxy.getImage();
            if (mediaImage != null) {
                InputImage image =
                    InputImage.fromMediaImage(mediaImage, rotationDegrees);

                initDrawingUtils(image);

                //START configuration of the facedetector
                FaceDetectorOptions realTimeOpts =
                    new FaceDetectorOptions.Builder()
                    .setContourMode(FaceDetectorOptions.CONTOUR_MODE_ALL)
                    .build();

                FaceDetector detector = FaceDetection.getClient(realTimeOpts);
                //END of configuration of face detector

                // START detector
                detector.process(image)
                    .addOnSuccessListener(
                        new OnSuccessListener < List < Face >> () {
                            @Override
                            public void onSuccess(List < Face > faces) {
                                if (!faces.isEmpty()) {
                                    //Get information about detected faces
                                    processFaces(faces);
                                } else {
                                    canvas.drawColor(Color.TRANSPARENT, PorterDuff.Mode.MULTIPLY);
                                    Log.i(TAG, "vuoto");
                                }
                            }
                        })
                    .addOnFailureListener(
                        e - > {
                            e.printStackTrace();
                        });
            }
        }
    });

    Log.i(TAG, "here4");

    //add preview and analysis to the lifecycle
    cameraProvider.bindToLifecycle((LifecycleOwner) this, cameraSelector, preview, imageAnalysis);

    Log.i(TAG, "here5");
}

the code never reaches the analyzer and so doesn't process the live frames. Could someone help me please?

Answer 1

Not sure if that's the main issue but it looks like you forgot to close your ImageProxy after you processed the image. Try to add this listener to the value returned from the process method:

.addOnCompleteListener(task -> imageProxy.close());

From the documentation:

If you are using the CameraX API, make sure to close the ImageProxy when finish using it, eg, by adding an OnCompleteListener to the Task returned from the process method. See the VisionProcessorBase class in the quickstart sample app for an example.

Answer 2

Little late answer, anyway I've encountered and solved the same problem today.

You should call the ImageProxy.close() at the end of the analyze method, this will allow for the next image to be delivered for analysis:

imageAnalysis.setAnalyzer(executor, new ImageAnalysis.Analyzer() {
    @Override
    public void analyze(@NonNull ImageProxy imageProxy) {
        int rotationDegrees = imageProxy.getImageInfo().getRotationDegrees();

        @SuppressLint("UnsafeOptInUsageError") 
        Image mediaImage = imageProxy.getImage();
        if (mediaImage != null) {
            ...
        }

        imageProxy.close() // <-- Add this line
    }
}