How do I transform an image when using Open GL Framebuffer in Android

Question

I'm using a live streaming library called Cine.io to save a frame from the stream to Android's external storage.

The library itself uses Google's Grafika resource.

When I simply set up a ByteBufferand then read it with GLReadPixel, the image is saved properly but it is inverted since GL images have the reverse coordinates of BMP coordinates.

In order to resolve this issue, I believe the most effective solution is to use an open GL framebuffer to render the incoming frame, flip it with an ortho matrix, and then read it with GLReadPixel in order to save it to disk.

This is the code that I believe will achieve this goal:

   int width = getWidth();
    int height = getHeight();
    ByteBuffer buf = ByteBuffer.allocateDirect(width * height * 4);
    buf.order(ByteOrder.LITTLE_ENDIAN);


    int[] fboId = new int[1];
    int[] rendId = new int[1];
    float[] projMatrix = new float[16];

    GLES20.glGenFramebuffers(1, fboId, 0);
    GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fboId[0]);

         GLES20.glGenRenderbuffers(1, rendId, 0);
    GLES20.glBindRenderbuffer(GLES20.GL_RENDERBUFFER, rendId[0]);
    GLES20.glRenderbufferStorage(GLES20.GL_RENDERBUFFER,GLES20.GL_RGBA4, width, height);


    switch(GLES20.glCheckFramebufferStatus(GLES20.GL_FRAMEBUFFER)){
        case GLES20.GL_FRAMEBUFFER_COMPLETE:
            Log.i(TAG, "Is Complete");
            break;
        case GLES20.GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT:
            Log.i(TAG, "Incomplete Attachment");
            break;
        case GLES20.GL_FRAMEBUFFER_INCOMPLETE_MISSING_ATTACHMENT:
            Log.i(TAG, "Missing Attachment");
            break;
        default: Log.i(TAG, "Something else is going on");

    }
    GLES20.glFramebufferRenderbuffer(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_RENDERBUFFER,rendId[0] );
    GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fboId[0]);
    GLES20.glViewport(0,0,width, height);
    android.opengl.Matrix.orthoM(projMatrix, 0, 0, width, height, 0, -1, 1);


  GLES20.glReadPixels(0, 0, width, height,
            GLES20.GL_RGBA4, GLES20.GL_UNSIGNED_BYTE, buf);

    GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);




    GlUtil.checkGlError("glReadPixels");
    buf.rewind();

However the code simply generates a blank image file that is empty (size 3.6kb). In addition, the variable "buff"'s block consists of only 0's after the GLreadpixels method is called on it.

Finally, if I set BindFramebuffer to 0 (default bind) and place it prior to the GLReadPixels call, then the image does show up but still inverted - it's like the Framebuffer commands never interacted with it.

How can I solve this?

PS: a lot of folks are trying to answer a previous, more convoluted version of the question below - but the crux of it still holds.

Answer 1

You could always flip the buffer manually. I haven't worked with java's ByteBuffer so the following code may not work, but it's mainly just supposed to be an example.

//num channels being the amount of channels per pixel, 4 representing r, g, b, a
int num_channels = 4;
//allocate buffer size
ByteBuffer new_buf = ByteBuffer.allocateDirect(width * height * num_channels);
buf.order(ByteOrder.LITTLE_ENDIAN);

//calculate row size of image
int row_size = width * num_channels
//loop through the contents of the buffer and flip it
for (int y = 0; y < height; ++y) {
    for (int x = 0; x < row_size; ++x) {
        //put a row of bytes starting from the end of the buffer into the start of the new buffer
        new_buf.put(buf.get(((height - (y + 1)) - x) * row_size));
    }
}

However this is pretty slow and definitely not ideal in an update loop. Instead, in your render code you can change how the texture is projected onto the screen using a matrix. Try using glOrtho like so:

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(0.0f, window_width, window_height, 0.0f, -1.0f, 1.0f);

This should flip your textures when rendering. Try experimenting and messing around with different values to get a feel for it.