how to get my bytebuf to send my entire message Netty

Question

I everyone.

I ask you because i have a problem with the ByteBuf in the decoder of Netty.

I want to decode my message who arrives by a Server but the ByteBuf doesn't work as i would.

The problem is the ByteBuf don't take all the byte of the message.

I explain, i have a message who have a length of 1221 bytes (it's a example), but the buffer size is only of 64 bytes.

When i'm trying to read, the Buffer with my length and i have a error like this :

io.netty.handler.codec.DecoderException: java.lang.IndexOutOfBoundsException: readerIndex(117) + length(101) exceeds writerIndex(192): PooledUnsafeDirectByteBuf(ridx: 117, widx: 192, cap: 192)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:470) ~[netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:697) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.37.Final.jar:4.1.37.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]

I'm think Netty don't have the time to read everything and send only a partial message, but i don't now if i can configure Netty for he must wait until the message arrived in totallity.

If anyone can help me, i appreciate

For most help, i give you the code of the decoder

    int length = buffer.readInt();
    int messageType = buffer.readInt();


        Supplier<AbstractMessage> supplier = SUPPLIERS.get(messageType);
        if (supplier == null) {
            LOGGER.debug("This message type isn't supported: {}", messageType);
        } else {
            ByteBuf data = buffer.readBytes(length);
            if (messageType != 6) {
                AbstractMessage message = supplier.get();
                message.read(data, version);
                list.add(message);
                LOGGER.debug("{}", message);
            }
        }
    }

}

A message format is like this : MessageLength in 4 bytes (int) MessageType in 4 bytes (int) Data in n bytes (MessageLength size)

I give you the documentation i use for the interpretation Here .

Answer 1

You will need to write your own decoder by extending ByteToMessageDecoder and buffer until you received everything. As this is TCP you may receive the bytes in fragmented fashion so you need to assemble it again by yourself.

Something like this should work:

class MyDecoder extends ByteToMessageDecoder {

    @Override
    protected void decode(ChannelHandlerContext ctx, ByteBuf input, List<Object> out) {
        if (input.readableBytes() < 4) {
            // we need to have at least 4 bytes to read to be able to get the message length
            return;
        }
        int length = input.getInt(input.readerIndex());
        if (input.readableBytes() < 8 + length) {
            // ensure we have enough data so we can also read the message type and the whole message body
            return;
        }
        // skip the length now
        input.skipBytes(4);

        int messageType = input.readInt();

        Supplier<AbstractMessage> supplier = SUPPLIERS.get(messageType);
        if (supplier == null) {
            LOGGER.debug("This message type isn't supported: {}", messageType);
            input.skip(length);
        } else {
            if (messageType != 6) {
                ByteBuf data = buffer.readSlice(length);
                AbstractMessage message = supplier.get();
                message.read(data, version);
                list.add(message);
                LOGGER.debug("{}", message);
            }
        }
    }
}

Answer 2

Thank's a lot. I made changes but your code inspires me and solved my problem. I thought well i receveid fragment message but i don't have a lot of knowledges in buffer.

If this post can help others peoples, i put below the corrected code

if (length == 0) {
        length = buffer.readInt();
        messageType = buffer.readInt();
    }

    if (buffer.writerIndex() < length + buffer.readerIndex()) {
        // ensure we have enough data so we can also read the message type and the whole message body
        return;
    }

    if (!(messageType == MessageType.HEARTBEAT_REQ.getValue() || messageType == MessageType.HEARTBEAT_CONF.getValue())) {
        LOGGER.debug("The message type is : {}", messageType);
    }

    Supplier<AbstractMessage> supplier = SUPPLIERS.get(messageType);
    if (supplier == null) {
        LOGGER.debug("This message type isn't supported: {}", messageType);
        buffer.skipBytes(length);
    } else {
        ByteBuf data = buffer.readSlice(length);
        AbstractMessage message = supplier.get();
        message.read(data, version);
        list.add(message);
        if (messageType != 6) {
            LOGGER.debug("{}", message);
        }
        length = 0;
        messageType = 0;
    }

For note, length and messageType variable are now static in the Decoder

private static int length = 0;
private static int messageType = 0;