为什么“queueInputBuffer”中的“MediaCodec CodecException”只发生在 Android API 29 上？

Question

Basic Info

targetSdkVersion 28

Goal : The objective of the class is to resize a video before sending to my server.

Problem : The app crashes only on API 29, whether with real devices or using AVD. For example, the code works fine on Pixel 2 API 28 , but not on Pixel 2 API 29

Source Code : I use mostly the sample code from bigflake.com .

Error message (detailed log are atteched below):

E/AndroidRuntime: FATAL EXCEPTION: Thread-20
    Process: com.myapp.myapp, PID: 9189
    android.media.MediaCodec$CodecException: Error 0xfffffff3
        at android.media.MediaCodec.native_queueInputBuffer(Native Method)
        at android.media.MediaCodec.queueInputBuffer(MediaCodec.java:2450)

What I have tried Due to the limited log info of MediaCodec$CodecException , I tried with no success changing from MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface to MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Flexible .

---

The logs will show you the last two buffer cycles . I hope you'll see something of value.

Log of the one NOT working on *Pixel 2 API 29*

D/VideoResolutionChanger: no audio encoder output buffer
D/VideoResolutionChanger: loop: V(true){extracted:49(done:false) decoded:45(done:false) encoded:43(done:false)} A(true){extracted:45(done:false) decoded:44(done:false) encoded:1(done:false) pending:-1} muxing:true(V:0,A:1)
D/VideoResolutionChanger: video decoder: returned input buffer: 3
    video extractor: returned buffer of size 31007
    video extractor: returned buffer for time 4448177
D/VideoResolutionChanger: audio decoder: returned input buffer: 1
    audio extractor: returned buffer of size 32
    audio extractor: returned buffer for time 900000
D/VideoResolutionChanger: video decoder: returned output buffer: 5
    video decoder: returned buffer of size 1382400
    video decoder: returned buffer for time 4132500
D/VideoResolutionChanger: output surface: await new image
D/VideoResolutionChanger: output surface: draw image
D/VideoResolutionChanger: input surface: swap buffers
D/VideoResolutionChanger: video encoder: notified of new frame
    audio decoder: returned output buffer: 0
    audio decoder: returned buffer of size 320
D/VideoResolutionChanger: audio decoder: returned buffer for time 880000
    audio decoder: output buffer is now pending: -1
    audio decoder: attempting to process pending buffer: 0
D/VideoResolutionChanger: audio encoder: returned input buffer: 0
    audio decoder: processing pending buffer: 0
    audio decoder: pending buffer of size 320
    audio decoder: pending buffer for time 880000
D/VideoResolutionChanger: should have added track before processing output true
    video encoder: returned output buffer: 0
    video encoder: returned buffer of size 8234
    video encoder: returned buffer for time 3954177
D/VideoResolutionChanger: no audio encoder output buffer
D/VideoResolutionChanger: loop: V(true){extracted:50(done:false) decoded:46(done:false) encoded:44(done:false)} A(true){extracted:46(done:false) decoded:45(done:false) encoded:1(done:false) pending:-1} muxing:true(V:0,A:1)
D/VideoResolutionChanger: video decoder: returned input buffer: 4
    video extractor: returned buffer of size 30897
    video extractor: returned buffer for time 4519077
D/VideoResolutionChanger: video extractor: EOS
D/VideoResolutionChanger: releasing extractor, decoder, encoder, and muxer
D/SurfaceUtils: disconnecting from surface 0xc94d8008, reason disconnectFromSurface
D/MPEG4Writer: Video track stopping. Stop source
    Video track source stopping
    Video track source stopped
I/MPEG4Writer: Received total/0-length (44/0) buffers and encoded 44 frames. - Video
W/MPEG4Writer: 0-duration samples found: 1
I/MPEG4Writer: Received total/0-length (1/0) buffers and encoded 1 frames. - Audio
    Audio track drift time: 0 us
D/MPEG4Writer: Video track stopped. Stop source
D/MPEG4Writer: Audio track stopping. Stop source
    Audio track source stopping
    Audio track source stopped
    Audio track stopped. Stop source
    Duration from tracks range is [241451, 3994621] us
    Stopping writer thread
D/MPEG4Writer: 0 chunks are written in the last batch
D/MPEG4Writer: Writer thread stopped
I/MPEG4Writer: Ajust the moov start time from 0 us -> 0 us
D/MPEG4Writer: Video track stopping. Stop source
D/MPEG4Writer: Audio track stopping. Stop source
E/AndroidRuntime: FATAL EXCEPTION: Thread-20
    Process: com.myapp.myname, PID: 9189
    android.media.MediaCodec$CodecException: Error 0xfffffff3
        at android.media.MediaCodec.native_queueInputBuffer(Native Method)
        at android.media.MediaCodec.queueInputBuffer(MediaCodec.java:2450)
        at com.myapp.myname.utils.VideoResolutionChanger.doExtractDecodeEditEncodeMux(VideoResolutionChanger.java:543)

---

Log of the one working on *Pixel 2 API 28*

D/VideoResolutionChanger: no audio encoder output buffer
    loop: V(true){extracted:39(done:true) decoded:39(done:true) encoded:39(done:true)} A(true){extracted:147(done:false) decoded:146(done:false) encoded:5(done:false) pending:-1} muxing:true(V:0,A:1)
    audio decoder: returned input buffer: 3
    audio extractor: returned buffer of size 32
    audio extractor: returned buffer for time 2940000
D/VideoResolutionChanger: audio decoder: returned output buffer: 2
    audio decoder: returned buffer of size 320
    audio decoder: returned buffer for time 2920000
    audio decoder: output buffer is now pending: -1
    audio decoder: attempting to process pending buffer: 2
    audio encoder: returned input buffer: 2
    audio decoder: processing pending buffer: 2
    audio decoder: pending buffer of size 320
    audio decoder: pending buffer for time 2920000
D/VideoResolutionChanger: no audio encoder output buffer
D/VideoResolutionChanger: loop: V(true){extracted:39(done:true) decoded:39(done:true) encoded:39(done:true)} A(true){extracted:148(done:false) decoded:147(done:false) encoded:5(done:false) pending:-1} muxing:true(V:0,A:1)
D/VideoResolutionChanger: audio decoder: returned input buffer: 0
    audio extractor: returned buffer of size -1
    audio extractor: returned buffer for time -1
    audio extractor: EOS
D/VideoResolutionChanger: audio decoder: returned output buffer: 3
    audio decoder: returned buffer of size 320
    audio decoder: returned buffer for time 2940000
    audio decoder: output buffer is now pending: -1
    audio decoder: attempting to process pending buffer: 3
D/VideoResolutionChanger: audio encoder: returned input buffer: 3
    audio decoder: processing pending buffer: 3
    audio decoder: pending buffer of size 320
    audio decoder: pending buffer for time 2940000
D/VideoResolutionChanger: no audio encoder output buffer
D/VideoResolutionChanger: loop: V(true){extracted:39(done:true) decoded:39(done:true) encoded:39(done:true)} A(true){extracted:149(done:true) decoded:148(done:false) encoded:5(done:false) pending:-1} muxing:true(V:0,A:1)
    audio decoder: returned output buffer: 0
    audio decoder: returned buffer of size 0
    audio decoder: returned buffer for time 2880000
    audio decoder: output buffer is now pending: -1
    audio decoder: attempting to process pending buffer: 0
D/VideoResolutionChanger: audio encoder: returned input buffer: 0
    audio decoder: processing pending buffer: 0
    audio decoder: pending buffer of size 0
    audio decoder: pending buffer for time 2880000
D/VideoResolutionChanger: audio decoder: EOS
D/VideoResolutionChanger: should have added track before processing outputtrue
    audio encoder: returned output buffer: 0
    audio encoder: returned buffer of size 761
    audio encoder: returned buffer for time 2801451
    audio encoder: EOS
    encoded and decoded video frame counts should match39 39
    decoded frame count should be less than extracted frame count39 39
D/VideoResolutionChanger: no frame should be pending -1 -1
    releasing extractor, decoder, encoder, and muxer
D/SurfaceUtils: disconnecting from surface 0xd8b50808, reason disconnectFromSurface
D/MPEG4Writer: Video track stopping. Stop source
I/MPEG4Writer: Received total/0-length (6/0) buffers and encoded 6 frames. - Audio
D/MPEG4Writer: Video track source stopping
I/MPEG4Writer: Audio track drift time: 0 us
D/MPEG4Writer: Video track source stopped
I/MPEG4Writer: Received total/0-length (38/0) buffers and encoded 38 frames. - Video
D/MPEG4Writer: Video track stopped. Stop source
D/MPEG4Writer: Audio track stopping. Stop source
    Audio track source stopping
    Audio track source stopped
    Audio track stopped. Stop source
    Duration from tracks range is [3060067, 3302540] us
    Stopping writer thread
D/MPEG4Writer: 0 chunks are written in the last batch
D/MPEG4Writer: Writer thread stopped
I/MPEG4Writer: Ajust the moov start time from 0 us -> 0 us
D/MPEG4Writer: Video track stopping. Stop source
D/MPEG4Writer: Audio track stopping. Stop source

Answer 1

I think the correct way to go about it is to let it go another loop after xxxxExtractor.advance() returns false.

I modified my code to only mark an extractor done after the size returned by xxxxExtractor.readSampleData returns -1 That way you know the buffer you are sending to queueInputBuffer is empty.

So: xxxxExtractorDone = !xxxxExtractor.advance(); -> xxxxExtractorDone = !xxxxExtractor.advance() && size == -1;

Answer 2

I have also used the Bigflake code and ran into this with Android 10 (API 29).

The other linked issues are helpful, but don't directly answer the question as regards how to modify the Bigflake code. After spending a couple of hours, this is what I came up with.

For both video and audio, within doExtractDecodeEditEncodeMux , prior to where

                 xxxxxDecoder.queueInputBuffer(
                            decoderInputBufferIndex,
                            0,
                            0,
                            0,
                            MediaCodec.BUFFER_FLAG_END_OF_STREAM);

is called, you want to add the following line of code, which ensures that you use an empty buffer to send the EOS.

                    // wait forever for a new buffer
                    decoderInputBufferIndex = xxxxxDecoder.dequeueInputBuffer(-1);

There are two such instances, and the resulting code for video looks like this, with most of the context given:

                videoExtractorDone = !videoExtractor.advance();
                if (videoExtractorDone) {
                    if (VERBOSE) {
                        final String log = "Video extractor: EOS";
                        Log.d(TAG, log);
                        emitLog(log);
                    }
                    Log.i(TAG, "WAITING indefinitely until an input video buffer for EOS is available");
                    // wait forever for a new buffer
                    decoderInputBufferIndex = videoDecoder.dequeueInputBuffer(-1); // this is the actually important line
                    videoDecoder.queueInputBuffer(
                            decoderInputBufferIndex,
                            0,
                            0,
                            0,
                            MediaCodec.BUFFER_FLAG_END_OF_STREAM);
                            Log.i(TAG, "Successfully sent video EOS");
                }

Hopefully that helps anyone else stumbling into this.

Answer 3

I met the same problem and finally solved it. I used the asynctaskMode and codec callback.

while (!mAudioExtractorDone) {
                    int size = mAudioExtractor.readSampleData(decoderInputBuffer, 0);
                    long presentationTime = mAudioExtractor.getSampleTime();
                    MLog.i(TAG, "audio extractor: ronInputBufferAvailable() presentationTime = " + presentationTime);
                    if (size >= 0) {
                        codec.queueInputBuffer(index, 0, size, presentationTime, mAudioExtractor.getSampleFlags());
                    }else {
                        Log.e(TAG, "onInputBufferAvailable: " + "size < 0 = " + size);
                        codec.queueInputBuffer(index, 0, 0, 0L, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
                        mAudioExtractorDone = true;
                    }
                    mAudioExtractor.advance();
                    mAudioExtractedFrameCount++;
                    logState();

Send the BUFFER_FLAG_END_OF_STREAM when the size is below zero. Note the marker bits of the loop ( mAudioExtractorDone ).