Android OpenGL ES 2.0 multiple textures and Camera

Question

I am trying to blend two textures together using GLSurfaceView.Renderer and SurfaceTexture.OnFrameAvailableListener.

The first texture is a live camera preview SurfaceTexture which works fine by its own (quality is bad, but that's another thing :)

The second tetxure is just a bitmap image, which also works fine by its own.

For both textures I am using:

GLES20.glActiveTexture(GLES20.GL_TEXTUREX); //GL_TEXTURE0 and GL_TEXTURE1 respectively
GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, mTextureX);
GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR );
GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_NEAREST );
GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE); 

within onSurfaceCreated.

In my OnDrawFrame I have:

mTriangleVertices.position(TRIANGLE_VERTICES_DATA_POS_OFFSET);
GLES20.glEnableVertexAttribArray(maPositionHandle);
checkGlError("glEnableVertexAttribArray maPositionHandle");
GLES20.glVertexAttribPointer(maPositionHandle, 3, GLES20.GL_FLOAT, false,
            TRIANGLE_VERTICES_DATA_STRIDE_BYTES, mTriangleVertices);
checkGlError("glVertexAttribPointer maPosition");

mTriangleVertices.position(TRIANGLE_VERTICES_DATA_UV_OFFSET);
GLES20.glEnableVertexAttribArray(maTextureHandle);
checkGlError("glEnableVertexAttribArray maTextureHandle");
GLES20.glVertexAttribPointer(maTextureHandle, 3, GLES20.GL_FLOAT, false,
            TRIANGLE_VERTICES_DATA_STRIDE_BYTES, mTriangleVertices);
checkGlError("glVertexAttribPointer maTextureHandle");

Matrix.multiplyMM(mMVPMatrix, 0, mVMatrix, 0, mMMatrix, 0);  
Matrix.multiplyMM(mMVPMatrix, 0, mProjMatrix, 0, mMVPMatrix, 0);

GLES20.glUniformMatrix4fv(muMVPMatrixHandle, 1, false, mMVPMatrix, 0);
GLES20.glUniformMatrix4fv(muSTMatrixHandle, 1, false, mSTMatrix, 0);
GLES20.glUniform1f(muCRatioHandle, mCameraRatio);

GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);
checkGlError("glDrawArrays");

My shaders are:

private final String mVertexShader =
    "uniform mat4 uMVPMatrix;\n" +
    "uniform mat4 uSTMatrix;\n" +
    "uniform float uCRatio;\n" +
    "attribute vec4 aPosition;\n" +
    "attribute vec4 aTextureCoord;\n" +
    "varying vec2 vTextureCoord;\n" +
    "varying vec2 vTextureNormCoord;\n" + 
    "void main() {\n" +
    "  gl_Position = uMVPMatrix * aPosition;\n" +
    "  vTextureCoord = (uSTMatrix * aTextureCoord).xy;\n" +
    "  vTextureNormCoord = aTextureCoord.xy;\n" +
    "}\n";

and

 private final String mFragmentShader =    
        "#extension GL_OES_EGL_image_external : require\n" +     
        "precision mediump float;\n" +  
        "varying vec2 vTextureCoord;\n" +
        "varying vec2 vTextureNormCoord;\n" +
        "uniform samplerExternalOES sTexture;\n" +
        "uniform samplerExternalOES sTexture1;\n" +
        "void main() {\n" +
        "  vec4 temp_FragColor1 = texture2D(sTexture, vTextureCoord);\n" +
        "  vec4 temp_FragColor2= texture2D(sTexture1, vTextureCoord);\n" +
        "gl_FragColor = temp_FragColor1;\n" +
        "}\n";

The idea of what I am trying to do, is IN the fragment shader, to have

gl_FragColor = temp_FragColor1 + temp_FragColor2;

with the bitmap texture on top of the camera live preview (bitmap texture blend with the camera texture). So if a bitmap texture fragment has alpha less than 1, then camera texture's fragment should show. When I try this, I am getting a Fatal Signal 11.

Any tips, links or even code fragments would be much appreciated!

Thank you in advance

Answer 1

How are you loading the external texture with data from Bitmap? I would suggest using a 2D texture instead for overlay. Then, in order to load your image to texture you could use:

Bitmap bitmap1 = null;
InputStream stream = null;
try {
   stream = new FileInputStream("/mnt/sdcard/...jpg");
} catch (FileNotFoundException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
}
bitmap1 = BitmapFactory.decodeStream(stream);
GLUtils.texImage2D(GLES20.GL_TEXTURE_2D, 0, bitmap1, 0);
bitmap1.recycle();

And for mixing texture data inside fragment shader:

"gl_FragColor = mix(base, blend, blend.a);  \n";

where base is the data from camera texture and blend is the data from bitmap texture.