Android视频实时过滤

Question

I am writing a video player where I try to apply a filter on each decoded frame before showing it on the screen.

I use MediaCodec to extract a frame. The frames are decoded to a Surface created from SurfaceTexture , rendered (off-screen) into a pbuffer, extracted with glReadPixels()

I have used the ExtractMpegFramesTest as an example from this page:

http://bigflake.com/mediacodec/

At this point I have ByteBuffer with the extracted pixels, on which I do some post processing (converting to grayscale , or running edge detection etc.)

Having done that, I want to render the filtered frame on the screen. I could again encode it with the MediaCodec and use a VideoView to render it, but that way each frame is encoded and decoded unnecesarryly.

Is there an efficient way to render these frames on the screen?

Answer 1

The simple answer is: upload the pixels to a GLES texture, using glTexImage2D() , and render a quad.

Depending on your filtering, you may also want to consider performing the operations entirely in GLES. This is significantly faster, but a bit harder to pull off because the filters must be written in the fragment shader (GLSL).

You can find an example of shader-based image filtering in Grafika ( demo video here ), along with some uses of glTexImage2D() to send bitmap data to a texture.