My Android app suddenly closes

Question

I am working on a Augmented Reality Android app with OpenCV.

Now, when I run my app, it runs for a couple of seconds and then suddenly closes without warning and without error message in logcat. I think this has to do with RAM memory running out but I am having a very hard time figuring out what causes this. Also, when I check ddms at the moment my app will suddenly close, all processes (not only my app) start disappearing. And sometimes I get an error message like this in logcat:

cannot map BpMemoryHeap (binder=0x767e84c0), size=2883584, fd=53 (Out of memory)

This is how my OnPreviewFrame was setup:

1. Add some callback buffers:

    float bytesPerPix = ImageFormat.getBitsPerPixel(params.getPreviewFormat()) / 8.0f; int frame_byteSize = (int) ((size.width * size.height) * bytesPerPix); for(int i = 0; i< AppConfig.AMOUNT_PREVIEW_BUFFERS ; i++) { camera.addCallbackBuffer(new byte[frame_byteSize]); } 

2. set PreviewCallback on the camera:

    cameraRenderer.getCamera().setPreviewCallbackWithBuffer(virtualLayerRenderer); 

Here is my updateCameraPose function that is called in the OnPreviewFrame method:

public void updateCameraPose(byte[] frameData, Camera camera) {

    Size size = camera.getParameters().getPreviewSize();

    Mat frameImg = new Mat();
    Mat mYuv = new Mat( size.height + size.height/2, size.width, CvType.CV_8UC1 );
    mYuv.put( 0, 0, frameData );

    camera.addCallbackBuffer(frameData);

    Imgproc.cvtColor( mYuv, frameImg, Imgproc.COLOR_YUV2GRAY_NV21, 1);

    FindCameraPose task = new FindCameraPose();
    task.execute(frameImg, cameraPose);

}

This function calls an asynctask and executes it, here is the code of my asynctask:

private class FindCameraPose extends AsyncTask<Mat, Void, Mat> {

    @Override
    protected Mat doInBackground(Mat... params) {
        String t = cameraIntDistPath;
        getCameraPose( 
                params[0].getNativeObjAddr(), 
                t, 
                params[1].getNativeObjAddr());
        return params[1];
    }

    @Override
    protected void onPostExecute(Mat result) {
        super.onPostExecute(result);
        if(result != null && !result.empty())
            Log.d(TAG, "Camera Pose: "+result.dump());
    }
    private native boolean getCameraPose(long frameImagePtr, String cameraIntDistPath, long cameraPosePtr);

}

The asynctask then calls a native method that gets the camera pose, this is the code for this native method:

JNIEXPORT jboolean JNICALL Java_be_wouterfranken_arboardgame_rendering_tracking_CameraPose_00024FindCameraPose_getCameraPose
      (JNIEnv * env, jobject jobject, jlong frameImagePtr, jstring cameraIntDistPath, jlong cameraPosePtr)
{
    Mat *camPose = (Mat *) cameraPosePtr;
    Mat *frameImage = (Mat *) frameImagePtr;

    __android_log_print(ANDROID_LOG_DEBUG,APPNAME, "Mats setup ...");

    const char * path = env->GetStringUTFChars(cameraIntDistPath, NULL);

    __android_log_print(ANDROID_LOG_DEBUG,APPNAME, "Path string setup started ... %s",path);

    Mat intrinsics;
    Mat distortion;

    __android_log_print(ANDROID_LOG_DEBUG,APPNAME, "Search for cameraPose started...");

    Utilities::loadCameraCalibration(path, intrinsics, distortion);
    env->ReleaseStringUTFChars(cameraIntDistPath, path);

    __android_log_print(ANDROID_LOG_DEBUG,APPNAME, "Camera calibration done");

    bool patternFound = pd.findPattern(*frameImage, pti);

    __android_log_print(ANDROID_LOG_DEBUG,APPNAME, "Patterns found? %d",patternFound);

    if(patternFound) {
        pti.computePose(p, intrinsics, distortion);
        *camPose = pti.pose3d;
        __android_log_print(ANDROID_LOG_DEBUG,APPNAME, "Pose computed");
    }

    return patternFound;
}

Any ideas?

UPDATE

Ok so I narrowed the memory leak down to a leak in the native heap, that is coming from the opencv_java library. Especially when I turn off the call to detect keypoints, the leak seems to be gone. I hope this is not a bug in OpenCV but it seems like that now. Also, even if I try to use another detector (SIFT/SURF/ORB/BRISK), this doesn't matter, still a memory leak.

Anyone ideas to what this could be related?

Answer 1

Have you tried to put the "android:largeHeap="true" parameter inside your Application tag inside your AndroidManifest.xml ?

Something like this:

<application android:label="@string/app_name" android:largeHeap="true">

<activity android:name=".MainActivity" android:label="@string/app_name">
    <intent-filter>
        <action android:name="android.intent.action.MAIN" />
        <category android:name="android.intent.category.LAUNCHER" />
    </intent-filter>
</activity>

Answer 2

Ok, so I just answered my own question.

I gave the callback buffer back to the camera before my calculation was done and this on every frame. This gave the camera the possibility to run the getCameraPose method on almost every frame (which is of course a lot). This called the tracking code almost every frame and thus more work is added than can be solved and this explains the massive amount of used memory in the heap.

From now on I'll make sure I only give my buffer back to the camera after the frame is completely handled.