Memory issue in netty

Question

I am using Netty 4.0.32. I have allocated MaxDirectMemorySize:256M

My BootStrap is like this:

bootStrap = new ServerBootstrap();
childGroup = new NioEventLoopGroup();
bootStrap.localAddress(protocolConstant.getPort());
bootStrap.channel(NioServerSocketChannel.class);
bootStrap.group(PARENTGROUP, childGroup);
bootStrap.childHandler(new MailChannelInitializer());
bootStrap.option(ChannelOption.SO_BACKLOG, BACKLOG);
bootStrap.childOption(ChannelOption.AUTO_READ, true);
bootStrap.childOption(ChannelOption.MAX_MESSAGES_PER_READ, 1 * 1024);
bootStrap.childOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, 512);
bootStrap.childOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, 256);
bootStrap.childOption(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT);

My pipeline looks like this:

SslHandler <-> ByteToMessageCodec(tranforms the ByteBuf to byte[] and vice versa) <-> BusinessLogicHandler

1) SslHandler for sslsupport. I am using java's SslEngine.

2) I extend the ByteToMessageCodec like this :

private class ByteConversionCodec extends ByteToMessageCodec<byte[]> {

    @Override
    protected void encode(ChannelHandlerContext ctx, byte[] msg, ByteBuf out)
            throws Exception {
        if (!ctx.channel().isWritable()) {
            ctx.flush();
        }
        out.writeBytes(msg);
    }

    @Override
    protected void decode(ChannelHandlerContext ctx, ByteBuf in,
            List<Object> out) throws Exception {
        byte[] arr;
        if (in.hasArray()) {
            arr = in.array();
        }
        else {
            arr = new byte[in.readableBytes()];
            in.readBytes(arr);
        }
        out.add(arr);
    }

}

If the channel is not writable when calling encode, I add a flush request so that the channel becomes writable again. Is this correct?

3) The BusinessLogicHandler designates the processing to a thread pool which does asyncprocessing(involves IO) and writes back to the pipeline using the SocketChannel object. All the writes are originated from the thread pool. And finally I add a flush() after all the writes so that all pending writes are flushed. Each write call involves a byte[] with a max size of 300bytes. All the writes add upto around 20Mb totally.

Thread [nioEventLoopGroup-3-1] (Suspended)  
    Thread.sleep(long) line: not available [native method]  
    Bits.reserveMemory(long, int) line: 651 
    DirectByteBuffer.<init>(int) line: 123  
    ByteBuffer.allocateDirect(int) line: 306    
    PoolArena$DirectArena.newChunk(int, int, int, int) line: 645    
    PoolArena$DirectArena(PoolArena<T>).allocateNormal(PooledByteBuf<T>, int, int) line: 228    
    PoolArena$DirectArena(PoolArena<T>).allocate(PoolThreadCache, PooledByteBuf<T>, int) line: 212  
    PoolArena$DirectArena(PoolArena<T>).allocate(PoolThreadCache, int, int) line: 132   
    PooledByteBufAllocator.newDirectBuffer(int, int) line: 271  
    PooledByteBufAllocator(AbstractByteBufAllocator).directBuffer(int, int) line: 155   
    PooledByteBufAllocator(AbstractByteBufAllocator).directBuffer(int) line: 146    
    PooledByteBufAllocator(AbstractByteBufAllocator).buffer(int) line: 83   
    SslHandler.allocate(ChannelHandlerContext, int) line: 1504  
    SslHandler.allocateOutNetBuf(ChannelHandlerContext, int) line: 1514 
    SslHandler.wrap(ChannelHandlerContext, boolean) line: 517   
    SslHandler.flush(ChannelHandlerContext) line: 500   
    DefaultChannelHandlerContext(AbstractChannelHandlerContext).invokeFlush() line: 663 
    DefaultChannelHandlerContext(AbstractChannelHandlerContext).flush() line: 644   
    Server$MailChannelInitializer$ByteConversionCodec.encode(ChannelHandlerContext, byte[], ByteBuf) line: 134  
    Server$MailChannelInitializer$ByteConversionCodec.encode(ChannelHandlerContext, Object, ByteBuf) line: 1    
    ByteToMessageCodec$Encoder.encode(ChannelHandlerContext, I, ByteBuf) line: 168  
    ByteToMessageCodec$Encoder(MessageToByteEncoder<I>).write(ChannelHandlerContext, Object, ChannelPromise) line: 107  
    Server$MailChannelInitializer$ByteConversionCodec(ByteToMessageCodec<I>).write(ChannelHandlerContext, Object, ChannelPromise) line: 108 
    DefaultChannelHandlerContext(AbstractChannelHandlerContext).invokeWrite(Object, ChannelPromise) line: 633   
    AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext, Object, ChannelPromise) line: 32   
    AbstractChannelHandlerContext$WriteTask(AbstractChannelHandlerContext$AbstractWriteTask).write(AbstractChannelHandlerContext, Object, ChannelPromise) line: 908 
    AbstractChannelHandlerContext$WriteTask(AbstractChannelHandlerContext$AbstractWriteTask).run() line: 893    
    NioEventLoop(SingleThreadEventExecutor).runAllTasks(long) line: 358 
    NioEventLoop.run() line: 357    
    SingleThreadEventExecutor$2.run() line: 112 
    DefaultThreadFactory$DefaultRunnableDecorator.run() line: 137   
    FastThreadLocalThread(Thread).run() line: 745   

Because of this I find a delay in getting the content. The thread sleeps for a while to get the memory. Could anyone kindly help me with this situation? Thank you.

Answer 1

If the channel is not writable when calling encode, I add a flush request so that the channel becomes writable again. Is this correct?

That doesn't look right to me. Your encoder isn't trying to write to the channel, it should just be trying to byte[] into a ByteBuf . Your BusinessLogicHandler already made the decision to write or not and whether to flush or not.

Have you tried removing this code to see if it fixes your problem?

if (!ctx.channel().isWritable()) {
    ctx.flush();
}