TextureView with MediaCodec decoder for H264 streams

Question

This is a follow-up question of this question .

This is my TextureView code:

public class VideoTextureView extends TextureView implements SurfaceTextureListener{

    private static final String LOG_TAG = VideoTextureView.class.getSimpleName();
    private MediaCodecDecoder mMediaDecoder;
    private MediaCodecAsyncDecoder mMediaAsyncDecoder;

    public VideoTextureView(Context context, AttributeSet attrs) {
        super(context, attrs);
        setSurfaceTextureListener(this);
        Log.d(LOG_TAG, "Video texture created.");
    }

    @Override
    public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) {
        Log.d(LOG_TAG, "Surface Available: " + width + " x " + height);
        mMediaDecoder = new MediaCodecDecoder();
        mMediaDecoder.Start();
        mMediaDecoder.SetSurface(new Surface(getSurfaceTexture()));
    }

    @Override
    public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) {
        // TODO Auto-generated method stub

    }

    @Override
    public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
        mMediaDecoder.Stop();
        return false;
    }

    @Override
    public void onSurfaceTextureUpdated(SurfaceTexture surface) {
        // TODO Auto-generated method stub

    }

}

My question - Is my TextureView implementation okay for rendering H264 streams decoded by MediaCodec ? Or do I need to do EGL setup or anything else?

Thanks in advance!

Answer 1

I am currently using a TextureView for rendering multiple streams in one activity using collection view cells on android (Sorry for ios terminology there).

It works fine but the issue is when you for example rotate the device there will be surface_destroyed followed by surface_available. As i see you are correctly stopping and starting your decoder.

One thing i do in my Decoder is:

List<NaluSegment> segments = NaluParser.parseNaluSegments(buffer);
        for (NaluSegment segment : segments) {
            // ignore unspecified NAL units.
            if (segment.getType() != NaluType.UNSPECIFIED) {

                // Hold the parameter set for stop/start initialization speed
                if (segment.getType() == NaluType.PPS) {
                    lastParameterSet[0] = segment;
                } else if (segment.getType() == NaluType.SPS) {
                    lastParameterSet[1] = segment;
                } else if (segment.getType() == NaluType.CODED_SLICE_IDR) {
                    lastParameterSet[2] = segment;
                }

                // add to input queue
                naluSegmentQueue.add(segment);
            }
        }

I hold onto the last parameter sets and last keyframe and on start i fill the naluSegmentQueue with these first off to reduce the delay in video rendering.

Answer 2

My TextureView implementation is okay as I tried with SurfaceView too and found same result. And as @fadden said -

EGL setup is only required if you're rendering with GLES. TextureView combines a SurfaceTexture with a custom View, and does the GLES rendering for you. Which is why the View must be hardware-accelerated for TextureView to work.

Thanks to @fadden.