java bytebuffer write read append

Question

I'm trying to use ByteBuffer to parse a stream of data packets which I receive as byte arrays and may contain partial or multiple packets. The packets are of variable length so I need to parse the first two bytes to get the packet length and it seems that once I flip() the buffer to read the packet I cannot go back and append more data if the packet is not complete.

How can I write data to the buffer, read some of it to verify the contents and then append more data to it if necessary while preserving the read data?

class Foo {

    static final byte[] DATA1 = {0, 13, 2, 65, 88, 80, 55, 51, 52, 84, 82, 43, -111};
    static final byte[] DATA2 = {0, 21, 6, 2, 3, 67, 23, 71, -77, 24, -66, -12, -76, 98, 25, 61, 54, -20, 127, -118, 71};
    static final byte[] DATA3 = {0, 21, 5, 3, 5, 67};
    static final byte[] DATA4 = {23, 72, -50, 24, -66, -9, -31, -86, 25, 61, -95, 75, 71, 47, -102, 0, 13, 2, 84};
    static final byte[] DATA5 = {82, 65, 28, 10, 65, 71, 52, 44, 11};

    ByteBuffer mByteBuffer = ByteBuffer.allocate(4096);

    public void test() {
        mByteBuffer.order(ByteOrder.BIG_ENDIAN);
        parse(DATA1);
        parse(DATA2);
        parse(DATA3);
        parse(DATA4);
        parse(DATA5);
    }

    public void parse(byte[] bytes) {
        short byteCount;
        short crc;

        mByteBuffer.put(bytes);
        mByteBuffer.flip();

        while (mByteBuffer.remaining() > 5) {
            byteCount = mByteBuffer.getShort();

            // check for invalid buffer
            if (byteCount < 5) {
                mByteBuffer.clear();
                return;
            }

            if (mByteBuffer.remaining() < (byteCount - 2)) {
                // TODO: incomplete packet, do something
            }

            // payload (u8[1 .. 4092])
            byte[] payload = new byte[byteCount - 4];
            mByteBuffer.get(payload, 0, payload.length);

            // crc (u16)
            crc = mByteBuffer.getShort();
        }

        mByteBuffer.clear();
    }
}

Answer 1

在getShort()之后或通常在任何获取或写入序列之后以及下一个read()之前，调用compact() read() 。

Answer 2

I've made quite a few changes here and they are all explained in the comments.

private final ByteBuffer mByteBufferAlloc = ByteBuffer.allocate( 4096 );
private ByteBuffer mByteBuffer = mByteBufferAlloc.duplicate(); // create a duplicate so we can use without affecting the original allocated buffer so we have a blank slate to reset to

public void test()
{
    mByteBuffer.order( ByteOrder.BIG_ENDIAN );
    parse( DATA1 );
    parse( DATA2 );
    parse( DATA3 );
    parse( DATA4 );
    parse( DATA5 );
}

public void parse( byte[] bytes )
{
    short byteCount;
    short crc;

    if( bytes.length > mByteBuffer.remaining() ) // Ran out of room, compact anything not yet processed back to the beginning of the buffer and start back with a copy of the allocated buffer
    {
        int numBytes = mByteBuffer.position();
        if( numBytes > 0 )
        {
            byte[] toCompact = new byte[numBytes];
            ((ByteBuffer) mByteBuffer.flip()).get( toCompact );
            mByteBuffer = mByteBufferAlloc.duplicate().put( toCompact );
        }
        else // nothing in the buffer, simply start fresh
            mByteBuffer = mByteBufferAlloc.duplicate();
    }
    mByteBuffer.put( bytes );
    mByteBuffer.flip();

    while( mByteBuffer.remaining() > 5 )
    {
        byteCount = mByteBuffer.getShort();

        // check for invalid buffer
        if( byteCount < 5 )
        {
            System.err.println( "Invalid data, throwing away buffer." );
            mByteBuffer.clear(); // something was invalid, throw away the entire buffer, but we should probably log an error or something
            return;
        }

        if( mByteBuffer.remaining() < (byteCount - 2) )
        {
            // not enough data, wait for more
            break;
        }

        // payload (u8[1 .. 4092])
        byte[] payload = new byte[byteCount - 4];
        mByteBuffer.get( payload, 0, payload.length );

        // crc (u16)
        crc = mByteBuffer.getShort();

        int pos = mByteBuffer.position(), limit = mByteBuffer.limit(), capacity = mByteBuffer.capacity();
        mByteBuffer = ((ByteBuffer) mByteBuffer.limit( capacity )).slice();
        mByteBuffer.limit( limit - pos ); // cut the packet off the beginning so we won't parse it again

        // TODO do something with the packet
    }

    mByteBuffer.position( mByteBuffer.limit() ).limit( mByteBuffer.capacity() ); // reset the position to the end of the added data and the limit to the capacity
    // mByteBuffer.clear(); Don't want to do this because we want the next call to append to the buffer so retain state

}