How can I correctly draw bouding box on detected face In Firebase ML kit Android?

Question

I'm working on an application for smile detection in live camera using CameraX and Firebase ML kit. The problem I'm facing is that I'm unable to draw bounding box on detected face. I've translated the points returned from bounding box to my view but it doesn't work because the image given to Firebase model is rotated which is different from what the image is in view.

Code for translating and drawing bounding box:

    // Draws a bounding box around the face.
    float left= (float) 0.0;
    float right=(float) 0.0;
    if(facing== LENS_FACING_FRONT){
        left=canvas.getWidth()-translateX(boundingBox.left);
        right=canvas.getWidth()-translateX(boundingBox.right);
    }
    else if(facing== CameraSelector.LENS_FACING_BACK) {
        left = translateX(boundingBox.left);
        right =translateX(boundingBox.right);
    }
    float top =translateY(boundingBox.top);
    float bottom=translateY(boundingBox.bottom);


    canvas.drawRect(left,top,right,bottom,boxPaint);

Code for image sent to firebase model:

FirebaseVisionImage image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation1);

The rotation is calculated:

CameraManager cameraManager = (CameraManager) context.getSystemService(CAMERA_SERVICE);
            int sensorOrientation=0 ;
            try {
                sensorOrientation = cameraManager
                        .getCameraCharacteristics(String.valueOf(lensFacing))
                        .get(CameraCharacteristics.SENSOR_ORIENTATION);
            } catch (CameraAccessException e) {
                e.printStackTrace();
            }

            int adjustedorientation;
            adjustedorientation=(orientation/90)*90;
            adjustedorientation=adjustedorientation+sensorOrientation;

            rotation1=(4-((adjustedorientation%360)/90))%4;

Answer 1

We made some changes to Firebase ML Kit to better distinguish the on-device APIs from cloud based APIs. " ML Kit "(without firebase branding) contains all the on-device APIs. Here's the migration guide from firebase mlkit to mlkit. All further improvements and new APIs will be released only with the new ML Kit. To use mlkit facedetection with CameraX, you can check the sample code here( https://github.com/googlesamples/mlkit/blob/master/android/vision-quickstart/app/src/main/java/com/google/mlkit/vision/demo/java/CameraXLivePreviewActivity.java ) to see how to draw the overlay for the face.

Answer 2

Good and simple example here: https://github.com/happysingh23828/CameraX-FaceDetection-MlKit