Android，将Mediacodec与libstreaming一起使用

Question

I've a problem with this library

https://github.com/fyhertz/libstreaming

it allows to send via wireless the streaming of photocamera, it use 3 methods: two with mediacodec and one with mediarecorder. I would like to modify it, and I have to use only the mediacodec;however first of all I tried the code of the example 2 of the library, but I've always found the same error: the log tell me that the device can use the mediacodec, it set the encoder and when it test the decoder it fall and the buffer is filled with -1.

This is the method in the EncoderDebugger class where the exception occurs, some kind soul can help me please?

private long decode(boolean withPrefix) {
    int n =3, i = 0, j = 0;
    long elapsed = 0, now = timestamp();
    int decInputIndex = 0, decOutputIndex = 0;
    ByteBuffer[] decInputBuffers = mDecoder.getInputBuffers();
    ByteBuffer[] decOutputBuffers = mDecoder.getOutputBuffers();
    BufferInfo info = new BufferInfo();

    while (elapsed<3000000) {

        // Feeds the decoder with a NAL unit
        if (i<NB_ENCODED) {

            decInputIndex = mDecoder.dequeueInputBuffer(1000000/FRAMERATE);
            if (decInputIndex>=0) {
                int l1 = decInputBuffers[decInputIndex].capacity();
                int l2 = mVideo[i].length;
                decInputBuffers[decInputIndex].clear();

                if ((withPrefix && hasPrefix(mVideo[i])) || (!withPrefix && !hasPrefix(mVideo[i]))) {

                    check(l1>=l2, "The decoder input buffer is not big enough (nal="+l2+", capacity="+l1+").");
                    decInputBuffers[decInputIndex].put(mVideo[i],0,mVideo[i].length);
                } else if (withPrefix && !hasPrefix(mVideo[i])) {

                    check(l1>=l2+4, "The decoder input buffer is not big enough (nal="+(l2+4)+", capacity="+l1+").");
                    decInputBuffers[decInputIndex].put(new byte[] {0,0,0,1});
                    decInputBuffers[decInputIndex].put(mVideo[i],0,mVideo[i].length);
                } else if (!withPrefix && hasPrefix(mVideo[i])) {

                    check(l1>=l2-4, "The decoder input buffer is not big enough (nal="+(l2-4)+", capacity="+l1+").");
                    decInputBuffers[decInputIndex].put(mVideo[i],4,mVideo[i].length-4);
                }

                mDecoder.queueInputBuffer(decInputIndex, 0, l2, timestamp(), 0);
                i++;
            } else {
                if (VERBOSE) Log.d(TAG,"No buffer available !7");
            }
        }

        // Tries to get a decoded image

        decOutputIndex = mDecoder.dequeueOutputBuffer(info, 1000000/FRAMERATE);
        if (decOutputIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
            decOutputBuffers = mDecoder.getOutputBuffers();
        } else if (decOutputIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
            mDecOutputFormat = mDecoder.getOutputFormat();
        } else if (decOutputIndex>=0) {
            if (n>2) {
                // We have successfully encoded and decoded an image !
                int length = info.size;
                mDecodedVideo[j] = new byte[length];
                decOutputBuffers[decOutputIndex].clear();
                decOutputBuffers[decOutputIndex].get(mDecodedVideo[j], 0, length);
                // Converts the decoded frame to NV21
                convertToNV21(j);
                if (j>=NB_DECODED-1) {

                    flushMediaCodec(mDecoder);
                    if (VERBOSE) Log.v(TAG, "Decoding "+n+" frames took "+elapsed/1000+" ms");
                    return elapsed;
                }
                j++;
            }
            mDecoder.releaseOutputBuffer(decOutputIndex, false);
            n++;
        }   
        elapsed = timestamp() - now;
    }

    throw new RuntimeException("The decoder did not decode anything.");

}

Answer 1

Here's my suggestions:

(1) check the settings of encoder and decoder, and make sure that they match. For example, revolution and color format are the same.

(2) make sure the very first packet generated by the encoder has been sent and pushed into the decoder. This packet defines the basic settings of the video stream.

(3) the decoder usually buffers 5-10 frames. So data in the buffer is invalid for a few hundred ms.

(4) while initiating the decoder, set the surface as null. Otherwise the output buffer will be read by the surface and probably released automatically.