Draw rect over camera preview

Question

I have a camera preview where i get every frame, for each frame i analyze the objects inside and i have a list that contain every object recognized and the location, so i have already the locations for every single object. Now i want to draw a rect around that object on camera preview, but doing canvas.drawRect doesn't work, any suggestion?

                for (final Detector.Recognition result : results) {
                final RectF location = result.getLocation();
                if (location != null && result.getConfidence() >= 0.1) {
                    result.setLocation(location);
                    mappedRecognitions.add(result);
                    Log.d("OBJECT: ", result.getTitle());

                    final Paint paint = new Paint();
                    paint.setColor(Color.RED);
                    paint.setStyle(Paint.Style.STROKE);
                    paint.setStrokeWidth(2.0f);
                    canvas_crop.drawRect(location, paint);
                    cropToFrameTransform.mapRect(location);
                }
            }

This is the layout XML

<androidx.coordinatorlayout.widget.CoordinatorLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity">

<RelativeLayout
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:background="@android:color/black"
    android:orientation="vertical">


    <FrameLayout
        android:id="@+id/container"
        android:layout_width="match_parent"
        android:layout_height="match_parent">

    </FrameLayout>

    <com.otaliastudios.cameraview.CameraView
        android:id="@+id/cameraView"
        android:layout_width="match_parent"
        android:layout_height="match_parent"/>
</RelativeLayout>


</androidx.coordinatorlayout.widget.CoordinatorLayout>

Answer 1

If you don't need the rectangles to be recorded, so, are only displayed in the live preview, then in general the solution is to overlay a custom view dedicated exclusively to rendering of drawings.

Create a custom view and place it on top, ensuring it always matches the position and size (aspect ratio) of the preview shown by the CameraView. Next an example to get you started, although you will need to add the logic to ensure it matches the CameraView "preview" metrics. Use the setTargets to pass the rectangles to paint:

Java:

public final class OverlayView extends View {
    private final Paint paint = new Paint();
    private final List<Rect> targets = new ArrayList<>();

    public OverlayView(@NonNull final Context context) {
        this(context, null);
    }

    public OverlayView(@NonNull final Context context, @Nullable final AttributeSet attrs) {
        this(context, attrs, 0);
    }

    public OverlayView(@NonNull final Context context, @Nullable final AttributeSet attrs, final int defStyleAttr) {
        super(context, attrs, defStyleAttr);

        final float density = context.getResources().getDisplayMetrics().density;
        this.paint.setStrokeWidth(2.0f * density);
        this.paint.setColor(Color.BLUE);
        this.paint.setStyle(Paint.Style.STROKE);
    }

    @Override
    protected void onDraw(final Canvas canvas) {
        super.onDraw(canvas);

        synchronized (this) {
            for (final Rect entry : this.targets) {
                canvas.drawRect(entry, this.paint);
            }
        }
    }

    public void setTargets(@NonNull final List<Rect> sources) {
        synchronized (this) {
            this.targets.clear();
            this.targets.addAll(sources);
            this.postInvalidate();
        }
    }
}

Kotlin:

class OverlayView(context: Context, attrs: AttributeSet) : View(context, attrs) {
    private val paint = Paint()
    private val targets: MutableList<Rect> = ArrayList()

    override fun onDraw(canvas: Canvas) {
        super.onDraw(canvas)

        synchronized(this) {
            for (entry in targets) {
                canvas.drawRect(entry, paint)
            }
        }
    }

    fun setTargets(sources: List<Rect>) {
        synchronized(this) {
            targets.clear()
            targets.addAll(sources)
            this.postInvalidate()
        }
    }

    init {
        val density = context.resources.displayMetrics.density
        paint.strokeWidth = 2.0f * density
        paint.color = Color.BLUE
        paint.style = Paint.Style.STROKE
    }
}

And for your specific xml, would be as next:

<androidx.coordinatorlayout.widget.CoordinatorLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity">

<RelativeLayout
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:background="@android:color/black"
    android:orientation="vertical">


    <FrameLayout
        android:id="@+id/container"
        android:layout_width="match_parent"
        android:layout_height="match_parent">

    </FrameLayout>

    <com.otaliastudios.cameraview.CameraView
        android:id="@+id/cameraView"
        android:layout_width="match_parent"
        android:layout_height="match_parent"/>
        
    <com.otaliastudios.cameraview.OverlayView 
        android:id="@+id/overlayView"
        android:layout_width="match_parent"
        android:layout_height="match_parent"/>
        
</RelativeLayout>


</androidx.coordinatorlayout.widget.CoordinatorLayout>

Note that you are using match_parent for the CameraView, this should be handled better in code to adjust into all possible camera aspect ratios, and consequently when you resolve the preview size, such must be also applied to the new OverlayView .