MediaCodec not decoding H264 encoded video data

Question

I'm developing an application that receives H264 encoded data over RTP and I am having troubles getting Android's MediaCodec to output anything. I'm depacketizing the RTP packets as described here https://stackoverflow.com/a/7668578/10788248

After encoded frames have been reassembled I feed them into the dequeued input buffers.

I don't get any errors when I queue the input buffers, but the method onOutputBufferAvailable is never called by the decoder's callback.

The only way I'm able to get it called is by passing the end-of-stream flag, and then the output size is 0.

My questions are, 'is there anything obviously wrong that I'm missing?' and 'what potential issues could lead to output buffers never becoming available, but the codec not throwing an error?'

Code Updated

LinkedList<DatagramPacket> packets = new LinkedList<>();
Thread socketReader = new Thread(() -> {
        try {
            DatagramSocket socket  = new DatagramSocket(videoPort);
            socket.connect(InetAddress.getByName(remoteAddress),remotePort);
            byte[] b;
            while (true){
                b = new byte[1500];
                DatagramPacket p = new DatagramPacket(b,1500);
                socket.receive(p);
                //Log.d(TAG,"RTP: "+bytesToHex(p.getData()));
                packets.add(p);
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    });
@SuppressLint({"NewApi", "LocalSuppress"}) Thread packetHandler = new Thread(() -> {
        try {
            MediaCodec decoder = MediaCodec.createDecoderByType(MIME_TYPE);
            MediaFormat decoderFormat = MediaFormat.createVideoFormat(MIME_TYPE, 1280, 720);
            LinkedList<ByteBuffer> decoderInputs = new LinkedList<>();
            LinkedList<Integer> decoderIndices = new LinkedList<>();

            decoder.setCallback(new MediaCodec.Callback() {
                @Override
                public void onInputBufferAvailable(@NonNull MediaCodec codec, int index) {
                    decoderInputs.add(codec.getInputBuffer(index));
                    decoderIndices.add(new Integer(index));
                }

                @Override
                public void onOutputBufferAvailable(@NonNull MediaCodec codec, int index, @NonNull MediaCodec.BufferInfo info) {
                    Log.d(TAG,"OUTPUT AVAILABLE!!!!");
                    Log.d(TAG, String.valueOf(info.size));
                }

                @Override
                public void onError(@NonNull MediaCodec codec, @NonNull MediaCodec.CodecException e) {
                    Log.e(TAG,e.toString());
                }
                @Override
                public void onOutputFormatChanged(@NonNull MediaCodec codec, @NonNull MediaFormat format) {
                    Log.d(TAG,"FORMAT CHANGED");
                }
            });
            decoder.configure(decoderFormat, null,null,0);
            decoder.start();
            RTPFrame frame = new RTPFrame();
            boolean sps = false;
            boolean pps = false;
            while (true){
                if(decoderIndices.peek()!=null){
                    DatagramPacket packet = packets.poll();
                    if(packet!=null){
                        byte[] data = packet.getData();
                        if(frame==null||(!frame.isInitialized())){
                            frame = new RTPFrame(data);
                        }
                        else{
                            if(frame.isComplete()){
                                Integer index = null;
                                ByteBuffer decoderInput = null;
                                try {
                                    decoderInput = decoderInputs.poll().put(frame.getFrame());
                                    index = decoderIndices.poll();
                                    int size = frame.getFrameSize();
                                    if (frame.SPS) {
                                        Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
                                        decoder.queueInputBuffer(index, 0, size, frame.getTime(), MediaCodec.BUFFER_FLAG_CODEC_CONFIG);
                                        sps = true;
                                        pps = false;
                                    } else if (frame.PPS) {
                                        if(sps){
                                            Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
                                            decoder.queueInputBuffer(index, 0, size, frame.getTime(), MediaCodec.BUFFER_FLAG_CODEC_CONFIG);
                                            pps = true;
                                        }
                                    } else if (!frame.badFrame) {
                                        if(sps&&pps){
                                            Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
                                            decoder.queueInputBuffer(index, 0, size, frame.getTime(), 0);
                                        }
                                    }
                                    else{
                                        throw new RuntimeException();
                                    }
                                }
                                catch(Exception e){
                                    e.printStackTrace();
                                    if(index!=null){
                                        decoderIndices.push(index);
                                    }
                                    decoderInput.clear();
                                    decoderInputs.push(decoderInput);
                                }
                                frame = new RTPFrame();
                            }
                            else{
                                frame.addNALUnit(data);
                            }
                        }
                        if(frame.isComplete()){
                            Integer index = null;
                            ByteBuffer decoderInput = null;
                            try {
                                decoderInput = decoderInputs.poll().put(frame.getFrame());
                                index = decoderIndices.poll();
                                int size = frame.getFrameSize();
                                if (frame.SPS) {
                                    Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
                                    decoder.queueInputBuffer(index, 0, size, frame.getTime(), MediaCodec.BUFFER_FLAG_CODEC_CONFIG);
                                    sps = true;
                                    pps = false;
                                } else if (frame.PPS) {
                                    if(sps){
                                        Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
                                        decoder.queueInputBuffer(index, 0, size, frame.getTime(), MediaCodec.BUFFER_FLAG_CODEC_CONFIG);
                                        pps = true;
                                    }
                                } else if (!frame.badFrame) {
                                    if(sps&&pps){
                                        Log.d(TAG,"Depacketized at "+Integer.toUnsignedString(frame.getTime())+" length = "+frame.getFrameSize()+": " + bytesToHex(frame.getFrame().array()));
                                        decoder.queueInputBuffer(index, 0, size, frame.getTime(), 0);
                                    }
                                }
                                else{
                                    throw new Exception("bad frame");
                                }
                            }
                            catch(Exception e){
                                e.printStackTrace();
                                if(index!=null){
                                    decoderIndices.push(index);
                                }
                                decoderInput.clear();
                                decoderInputs.push(decoderInput);
                            }
                            frame = new RTPFrame();
                        }
                    }
                }
            }
        } catch (Exception e) {e.printStackTrace();}
    });

Here's a sample of all the NAL units I received for 1 frame, and how I depacketized them.

RTP: 80 63 00 2D 27 0E 64 30 66 B4 BA 42 3C 81 E0 00 80 6F...

RTP: 80 63 00 2E 27 0E 64 30 66 B4 BA 42 3C 01 F3 0F E0 3F...

RTP: 80 63 00 2F 27 0E 64 30 66 B4 BA 42 3C 01 37 9B FA BA...

RTP: 80 63 00 30 27 0E 64 30 66 B4 BA 42 3C 01 7F EA 75 7C...

RTP: 80 63 00 31 27 0E 64 30 66 B4 BA 42 3C 01 FA D8 A9 FF...

RTP: 80 63 00 32 27 0E 64 30 66 B4 BA 42 3C 01 1B C5 BC C0...

RTP: 80 63 00 33 27 0E 64 30 66 B4 BA 42 3C 01 0F F4 9A DE...

RTP: 80 63 00 34 27 0E 64 30 66 B4 BA 42 3C 01 F4 35 CD 28...

RTP: 80 63 00 35 27 0E 64 30 66 B4 BA 42 3C 01 9E 45 70 13...

RTP: 80 E3 00 36 27 0E 64 30 66 B4 BA 42 3C 41 0F 18 0D 83...

D/RTPreader: Depacketized at 2985639104 length = 12611: 00 00 00 01 21 E0 00 80 6F F0 B4 24 CD 5F 45 80 79 6E 0C...

Here's a sample SPS and PPS before and after depacketizing

RTP: 80 63 00 00 5F DF A1 70 4F 2F 8A 3E 27 42 00 1F 8D 68 05 00 5B A1 00 00 03 00 01 00 00 03 00 1E 0F 10 7A 80

Depacketized at 1608491376 length = 27: 00 00 01 27 42 00 1F 8D 68 05 00 5B A1 00 00 03 00 01 00 00 03 00 1E 0F 10 7A 80

Depacketized at 1608491376 length = 7: 00 00 01 28 CE 32 48

RTP: 80 63 00 01 5F DF A1 70 4F 2F 8A 3E 28 CE 32 48

Answer 1

I'd like to thank @greeble31 and @ChrisBe for their help in figuring out the issue. The problem was indeed with my RTPFrame class. Specifically, the method getFrame() , which iterated through a list of ByteBuffers containing NAL unit data and used put(ByteBuffer) to add them to bb . This incremented the position of bb until it was equal to limit .

public ByteBuffer getFrame() {
    size = 4;
    int nalCount = nalUnits.size();
    for (int i = 0;i<nalCount;i++){
        size+=nalUnits.get(i).data.length;
    }

    ByteBuffer bb = ByteBuffer.allocate(size);
    bb.put(new byte[]{0,0,0,1});
    for(NALUnit unit:nalUnits){
        bb.put(unit.data);
    }
    return bb;
}

Changing return bb; to return (ByteBuffer)bb.position(0); fixed the issue.