Tap to adjust focus / exposure in CameraX

Question

I would like to implement the standard UX of tapping a point on the preview to adjust the auto-focus and auto-exposure points to the place where they tapped. I found the Preview.focus() function, however it says that it needs dimensions in the "sensor coordinate frame", which I'm assuming is not the same as the TextureView's TouchEvent pixel coordinates.

How do I convert from the TextureView preview's touch coordinates to the "sensor coordinate frame" expected by Preview.focus()?

It would be great if this example was part of the sample code, as it seems like a pretty common use case that nearly everyone will expect.

Answer 1

TextureView's coordinates are not the same as sensor coordinates. Please refer to the sample codes here (Please note that the "CameraView" is not public in maven repository yet. So we do not encourage you use it now). We understand these are a lot of work so CameraX team is also developing a more developer-friendly version of focus/metering API.

The basic flow is as below: (1) get x, y from a view touch event. (2) Calculate the relative camera orientation using device orientation and camera2 CameraCharacteristics.SENSOR_ORIENTATION. The value represents the clockwise angle through which the sensor image needs to be rotated to be upright in current device orientation.
(3) swap x, y for 90 / 270 degree , and reverse the x, y properly by the orientation. reverse the x for mirroring (front camera) (4) transform to the sensor coordinates using CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE), view width/height .

note: for the camera id, for now you can get find first camera_id in mCameraManager.getCameraIdList() with correct lens facing. however the algorithm could be changed.

Answer 2

This blog post written by a Google engineer explains exactly how to do it (in Kotlin).

This is how to achieve the same in Java:

private void setUpTapToFocus() {
    textureView.setOnTouchListener(new OnTouchListener() {
        @Override
        public boolean onTouch(View v, MotionEvent event) {
            if (event.getAction() != MotionEvent.ACTION_UP) {
                /* Original post returns false here, but in my experience this makes
                onTouch not being triggered for ACTION_UP event */
                return true;
            }
            TextureViewMeteringPointFactory factory = new TextureViewMeteringPointFactory(textureView);
            MeteringPoint point = factory.createPoint(event.getX(), event.getY());
            FocusMeteringAction action = FocusMeteringAction.Builder.from(point).build();    
            cameraControl.startFocusAndMetering(action);
            return true;
        }
    });
}

The cameraControl object can be instantiated like this:

CameraControl cameraControl = CameraX.getCameraControl(CameraX.LensFacing.BACK);

but make sure you have

implementation "androidx.camera:camera-view:1.0.0-alpha03"

within your build.gradle dependencies.

---

For reference, here's the original Kotlin code from Husayn Hakeem blog post :

private fun setUpTapToFocus() {
    textureView.setOnTouchListener { _, event ->
        if (event.action != MotionEvent.ACTION_UP) {
            return@setOnTouchListener false
        }

        val factory = TextureViewMeteringPointFactory(textureView)
        val point = factory.createPoint(event.x, event.y)
        val action = FocusMeteringAction.Builder.from(point).build()
        cameraControl.startFocusAndMetering(action)
        return@setOnTouchListener true
    }
}