Android Things摄像头流（带有opencv）

Question

Having problem with camera permission for android things, when I try to Run the simple OpenCV application, this message is shown "It seems that you device does not support camera (or it is locked), Application will be closed. Logcat - filter in Android Studio showing me that OpenCV library is loaded correctly, so I think maybe the problem is permission (you can check my AndroidManifest file) or maybe bad compatibility with OpenCV library...

Also Logcat report this error : "E/JavaCameraView: Camera #0failed to open: Fail to connect to camera service"

Link to source code: https://github.com/mrAlex-dev/CameraPreview-AndroidThings

PS I`m using: Raspberry Pi 3 B, Raspberry Pi Camera v1.3, Android Things - 0.4.1-devpreview, OpenCV-3.3.0-android-sdk...

Can someone help me please, I`m new at this.

Code for MainActivity.java:

package com.example.alexd.camerapreviewandroidthings;

import android.app.Activity;
import android.os.Bundle;
import android.util.Log;
import android.view.SurfaceView;

import org.opencv.android.BaseLoaderCallback;
import org.opencv.android.CameraBridgeViewBase;
import org.opencv.android.JavaCameraView;
import org.opencv.android.LoaderCallbackInterface;
import org.opencv.android.OpenCVLoader;
import org.opencv.core.CvType;
import org.opencv.core.Mat;

public class MainActivity extends Activity implements CameraBridgeViewBase.CvCameraViewListener2 {

    private static String TAG = "MainActivity";
    JavaCameraView javaCameraView;
    Mat mRgba;

    BaseLoaderCallback mLoaderCallBack = new BaseLoaderCallback(this) {
        @Override
        public void onManagerConnected(int status) {
            switch(status){
                case BaseLoaderCallback.SUCCESS:{
                    javaCameraView.enableView();
                    break;
                }
                default:{
                    super.onManagerConnected(status);
                    break;
                }
            }
            super.onManagerConnected(status);
        }
    };

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        javaCameraView = (JavaCameraView)findViewById(R.id.java_camera_view);
        javaCameraView.setVisibility(SurfaceView.VISIBLE);
        javaCameraView.setCvCameraViewListener(this);
    }

    @Override
    protected  void onPause(){
        super.onPause();
        if(javaCameraView!=null)
            javaCameraView.disableView();
    }

    @Override
    protected  void onDestroy(){
        super.onDestroy();
        if(javaCameraView!=null)
            javaCameraView.disableView();
    }

    @Override
    protected  void onResume(){
        super.onResume();
        if(OpenCVLoader.initDebug()){
            Log.i(TAG, "OpenCV loaded successfully");
            mLoaderCallBack.onManagerConnected(LoaderCallbackInterface.SUCCESS);
        }
        else{
            Log.i(TAG, "OpenCV not loaded");
            OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_3_0, this, mLoaderCallBack);
        }
    }

    @Override
    public void onCameraViewStarted(int width, int height) {
        mRgba = new Mat(height, width, CvType.CV_8UC4);
    }

    @Override
    public void onCameraViewStopped() {
        mRgba.release();
    }

    @Override
    public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
        mRgba = inputFrame.rgba();
        return mRgba;
    }
}

And this is AndroidManifest.xml:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.example.alexd.camerapreviewandroidthings">

    <uses-permission android:name="android.permission.CAMERA" />
    <uses-feature android:name="android.hardware.camera" />
    <uses-feature android:name="android.hardware.camera.autofocus" />
    <uses-feature android:name="android.hardware.camera2.*" />

    <uses-permission android:name="android.permission.WRITE_SETTINGS"/>
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="com.google.android.things.permission.MANAGE_INPUT_DRIVERS" />

    <application>
        <uses-library android:name="com.google.android.things" />

        <activity android:name=".MainActivity">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.IOT_LAUNCHER" />
                <category android:name="android.intent.category.DEFAULT" />
            </intent-filter>
        </activity>
    </application>

</manifest>

And this is Logcat errors:

09-13 08:50:29.812 1880-1880/? E/cutils-trace: Error opening trace file: No such file or directory (2)
09-13 08:50:30.026 1880-1880/? E/memtrack: Couldn't load memtrack module (No such file or directory)
09-13 08:50:30.026 1880-1880/? E/android.os.Debug: failed to load memtrack module: -2
09-13 08:50:32.090 1899-1899/? E/cutils-trace: Error opening trace file: No such file or directory (2)
09-13 08:50:32.300 1899-1899/? E/memtrack: Couldn't load memtrack module (No such file or directory)
09-13 08:50:32.300 1899-1899/? E/android.os.Debug: failed to load memtrack module: -2
09-13 08:50:35.356 1913-1913/? E/cutils-trace: Error opening trace file: No such file or directory (2)
09-13 08:50:35.573 1911-1911/? E/memtrack: Couldn't load memtrack module (No such file or directory)
09-13 08:50:35.573 1911-1911/? E/android.os.Debug: failed to load memtrack module: -2
09-13 08:50:37.811 150-150/? E/libEGL: called unimplemented OpenGL ES API
09-13 08:50:37.811 150-150/? E/SurfaceFlinger: glCheckFramebufferStatusOES error 0
09-13 08:50:37.812 150-150/? E/SurfaceFlinger: got GL_FRAMEBUFFER_COMPLETE_OES error while taking screenshot
09-13 09:00:00.843 412-429/system_process E/BatteryStatsService: no controller energy info supplied
09-13 09:00:00.848 412-429/system_process E/KernelWakelockReader: neither /proc/wakelocks nor /d/wakeup_sources exists
09-13 08:52:02.318 412-478/system_process E/TaskPersister: File error accessing recents directory (directory doesn't exist?).
09-13 08:51:58.902 155-155/? E/Camera2Client: notifyError: Error condition 0 reported by HAL, requestId -1
09-13 08:50:39.321 155-1838/? E/V4L2CameraHAL: virtual int v4l2_camera_hal::V4L2Camera::setupStreams(camera3_stream_configuration_t *):380: V4L2 only supports 1 stream configuration at a time (stream 0 is format 34, width 640, height 480, stream 1 is format 33, width 320, height 240).
09-13 08:50:39.321 155-1838/? E/Camera: int default_camera_hal::Camera::configureStreams(camera3_stream_configuration_t *):0: Failed to setup stream set
09-13 08:50:39.321 155-1838/? E/Camera3-Device: Camera 0: configureStreamsLocked: Set of requested inputs/outputs not supported by HAL
09-13 08:50:39.321 155-1838/? E/Camera3-Device: Camera 0: setUpRequestLocked: Can't set up streams: Invalid argument (-22)
09-13 08:50:39.321 155-1838/? E/Camera3-Device: Camera 0: convertMetadataListToRequestListLocked: Can't create capture request
09-13 08:50:39.321 155-1838/? E/Camera2-StreamingProcessor: startStream: Camera 0: Unable to set preview request to start preview: Invalid argument (-22)
09-13 08:50:39.321 155-1838/? E/Camera2Client: startPreviewL: Camera 0: Unable to start streaming preview: Invalid argument (-22)
09-13 08:50:39.262 162-1964/? E/FileSource: Failed to open file '/system/media/audio/ui/VideoRecord.ogg'. (No such file or directory)
09-13 08:50:39.262 162-1964/? E/GenericSource: Failed to create data source!
09-13 08:50:39.262 155-1838/? E/MediaPlayer: error (1, -2147483648)
09-13 08:50:39.266 162-1966/? E/FileSource: Failed to open file '/system/media/audio/ui/VideoStop.ogg'. (No such file or directory)
09-13 08:50:39.266 162-1966/? E/GenericSource: Failed to create data source!

Answer 1

The android.permission.CAMERA permission is a dangerous permission that would typically require granting at runtime by the user on standard Android. In Android Things, these permissions are granted automatically, but only on device boot. You need to do one of the following to give that permission to your app:

1. Reboot the device after the first installation. This will grant the permission on restart.

2. Install the APK manually using adb install -g <APK> the first time. This flag grants all requested permission on install, so you can avoid the reboot.

3. Upgrade to Android Studio 3.0, which will automatically grant these permission on install for you so you don't need to do either #1 or #2.

Answer 2

I'm working AndroidThings on Raspiberry 3B with a CSI camera. When I use Android Camera1 API try to open camera, I found some errors like

Can't Connect Camera using Android Camera (1) API on Raspberry Pi running AndroidThings

so I try to use Android Camera2 API. I got errors when I try to create preview & capture 2 streams session.I found this issue:37130806

It says The Raspberry Pi camera HAL is implemented using V4L2. Unfortunately the V4L2 interface is not perfectly compatible with the Android HAL requirements. Notably, V4L2 does not officially support multiple streams being configured at the same time.

So maybe that's the problem

Answer 3

as for my knowledge, android things currently don't support Graphic: hardware acceleration as it's not enabled by default. according to the question

 javaCameraView = (JavaCameraView)findViewById(R.id.java_camera_view);
    javaCameraView.setVisibility(SurfaceView.VISIBLE);
    javaCameraView.setCvCameraViewListener(this);

requires hardware accaleration on setting up a SurfaceView. check this it depends on OpenGL ES

so for the sake of getting it up and running is to implement something like follows.

        mCameraThread = new HandlerThread("CameraBackground");

    mCameraThread.start();

    mCameraHandler = new Handler(mCameraThread.getLooper());

    mCamera = RCamera.getInstance();

    mCamera.initializeCamera(this, mCameraHandler, mOnImageAvailableListener);

    takePicHandler = new Handler();

    takePicHandler.post(runnable);

and handlers and ImageReader.OnImageAvailableListener will be like

    private Runnable runnable = new Runnable() {
    @Override
    public void run() {
        Log.println(Log.INFO, "runnable", "running");

        mCamera.takePicture();

        takePicHandler.postDelayed(runnable, 0);

    }
};

private ImageReader.OnImageAvailableListener mOnImageAvailableListener =
        new ImageReader.OnImageAvailableListener() {
            @Override
            public void onImageAvailable(ImageReader reader) {
                Image image = reader.acquireLatestImage();
                // get image bytes
                ByteBuffer imageBuf = image.getPlanes()[0].getBuffer();

                final byte[] imageBytes = new byte[imageBuf.remaining()];

                imageBuf.get(imageBytes);

                image.close();

                onPictureTaken(imageBytes);
            }
        };

this will work just fine with android things and RPI3, cheers!!