netty 的非阻塞线程模型是如何工作的

Question

Currently, I'm reading the book "Reactive Programming with RxJava" by Tomasz Nurkiewicz . In chapter 5 he compares two different approaches to build an HTTP server wich one of them is based on a netty framework .

And I can't figure out how using such a framework can help to build more responsive server compare to the classic approach with a thread per request blocking IO.

The main concept is to utilize as few threads as possible but if there is some blocking IO operation such as DB access that means the very limited number on concurrent connection could be handled at a time

I've reproduced an example from that book.

Initializing the server:

public static void main(String[] args) throws Exception {
    EventLoopGroup bossGroup = new NioEventLoopGroup(1);
    EventLoopGroup workerGroup = new NioEventLoopGroup();
    try {
        new ServerBootstrap()
                .option(ChannelOption.SO_BACKLOG, 50_000)
                .group(bossGroup, workerGroup)
                .channel(NioServerSocketChannel.class)
                .childHandler(new HttpInitializer())
                .bind(8080)
                .sync()
                .channel()
                .closeFuture()
                .sync();
    } finally {
        bossGroup.shutdownGracefully();
        workerGroup.shutdownGracefully();
    }
}

The size of worker group thread pool is availableProcessors * 2 = 8 on my machine.

To simulate some IO operation and be able to see what is going on in the LOG I've added latency(but it could be some business logic invocation) of 1sec to the handler:

class HttpInitializer extends ChannelInitializer<SocketChannel> {

    private final HttpHandler httpHandler = new HttpHandler();

    @Override
    public void initChannel(SocketChannel ch) {
        ch
                .pipeline()
                .addLast(new HttpServerCodec())
                .addLast(httpHandler);
    }
}

And the handler itself:

class HttpHandler extends ChannelInboundHandlerAdapter {

    private static final Logger log = LoggerFactory.getLogger(HttpHandler.class);

    @Override
    public void channelReadComplete(ChannelHandlerContext ctx) {
        ctx.flush();
    }

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) {
        if (msg instanceof HttpRequest) {
            try {
                System.out.println(format("Request received on thread '%s' from '%s'", Thread.currentThread().getName(), ((NioSocketChannel)ctx.channel()).remoteAddress()));
            } catch (Exception ex) {}
            sendResponse(ctx);
        }
    }

    private void sendResponse(ChannelHandlerContext ctx) {
        final DefaultFullHttpResponse response = new DefaultFullHttpResponse(
                HTTP_1_1,
                HttpResponseStatus.OK,
                Unpooled.wrappedBuffer("OK".getBytes(UTF_8)));
        try {
            TimeUnit.SECONDS.sleep(1);
        } catch (Exception ex) {
            System.out.println("Ex catched " + ex);
        }
        response.headers().add("Content-length", 2);
        ctx.writeAndFlush(response);
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
        log.error("Error", cause);
        ctx.close();
    }
}

The client to simulate multiple concurrent connections:

public class NettyClient {

    public static void main(String[] args) throws Exception {
        NettyClient nettyClient = new NettyClient();
        for (int i = 0; i < 100; i++) {
            new Thread(() -> {
                try {
                    nettyClient.startClient();
                } catch (Exception ex) {
                }
            }).start();
        }
        TimeUnit.SECONDS.sleep(5);
    }

    public void startClient()
            throws IOException, InterruptedException {

        InetSocketAddress hostAddress = new InetSocketAddress("localhost", 8080);
        SocketChannel client = SocketChannel.open(hostAddress);

        System.out.println("Client... started");

        String threadName = Thread.currentThread().getName();

        // Send messages to server
        String[] messages = new String[]
                {"GET / HTTP/1.1\n" +
                        "Host: localhost:8080\n" +
                        "Connection: keep-alive\n" +
                        "Cache-Control: max-age=0\n" +
                        "Upgrade-Insecure-Requests: 1\n" +
                        "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36\n" +
                        "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3\n" +
                        "Accept-Encoding: gzip, deflate, br\n" +
                        "Accept-Language: ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7"};

        for (int i = 0; i < messages.length; i++) {
            byte[] message = new String(messages[i]).getBytes();
            ByteBuffer buffer = ByteBuffer.wrap(message);
            client.write(buffer);
            System.out.println(messages[i]);
            buffer.clear();
        }
        client.close();
    }
}

Expected -

Our case is the blue line with the only difference that delay was set to 0.1sec instead of 1sec as I explained above. With 100 concurrent connection, I was expecting 100 RPS because there were 90k RPS with 100k concurrent connection with 0.1 delays as the graph shows.

Actual - netty handles only 8 concurrent connection at a time, wait while sleep expires, take another bunch of 8 requests and so on. As a result, it took about 13sec to complete all requests. It's obvious to handle more clients I need to allocate more threads.

But this is exactly how the classic blocking IO approach works! Here the logs on the server-side, as you can see first 8 requests handled and one second later another 8 requests

2019-07-19T12:34:10.791Z Request received on thread 'nioEventLoopGroup-3-2' from '/127.0.0.1:49466'
2019-07-19T12:34:10.791Z Request received on thread 'nioEventLoopGroup-3-1' from '/127.0.0.1:49465'
2019-07-19T12:34:10.792Z Request received on thread 'nioEventLoopGroup-3-8' from '/127.0.0.1:49464'
2019-07-19T12:34:10.793Z Request received on thread 'nioEventLoopGroup-3-7' from '/127.0.0.1:49463'
2019-07-19T12:34:10.799Z Request received on thread 'nioEventLoopGroup-3-6' from '/127.0.0.1:49462'
2019-07-19T12:34:10.802Z Request received on thread 'nioEventLoopGroup-3-3' from '/127.0.0.1:49467'
2019-07-19T12:34:10.802Z Request received on thread 'nioEventLoopGroup-3-4' from '/127.0.0.1:49461'
2019-07-19T12:34:10.803Z Request received on thread 'nioEventLoopGroup-3-5' from '/127.0.0.1:49460'
2019-07-19T12:34:11.798Z Request received on thread 'nioEventLoopGroup-3-8' from '/127.0.0.1:49552'
2019-07-19T12:34:11.798Z Request received on thread 'nioEventLoopGroup-3-1' from '/127.0.0.1:49553'
2019-07-19T12:34:11.799Z Request received on thread 'nioEventLoopGroup-3-2' from '/127.0.0.1:49554'
2019-07-19T12:34:11.801Z Request received on thread 'nioEventLoopGroup-3-6' from '/127.0.0.1:49470'
2019-07-19T12:34:11.802Z Request received on thread 'nioEventLoopGroup-3-3' from '/127.0.0.1:49475'
2019-07-19T12:34:11.805Z Request received on thread 'nioEventLoopGroup-3-7' from '/127.0.0.1:49559'
2019-07-19T12:34:11.805Z Request received on thread 'nioEventLoopGroup-3-4' from '/127.0.0.1:49468'
2019-07-19T12:34:11.806Z Request received on thread 'nioEventLoopGroup-3-5' from '/127.0.0.1:49469'

So my question is - how could netty (or something similar) with its non-blocking and event-driven architecture utilize the CPU more effectively? If we only had 1 thread per each loop group the pipeline would be as follows:

1. ServerChannel selection key set to ON_ACCEPT
2. ServerChannel accept a connection and ClientChannel selection key set to ON_READ
3. Worker thread read the content of this ClientChannel and pass to the chain of handlers.
4. Even if the ServerChannel thread accept another client coonection and put it to some sort of queue, worker thread can't do anything before all handlers in the chain finish their job. From my the perspective of view thread can't just switch to another job since even waiting for the response from remote DB requires CPU ticks.

Answer 1

"how could netty (or something similar) with its non-blocking and event-driven architecture utilize the CPU more effectively? "

It cannot.

The goal of asynchronous (non-blocking and event-driven) programming is to save core memory, when tasks are used instead of threads as units of parallel work. This allows to have millions of parallel activities instead of thousands.

CPU cycles cannot be saved automatically - it is always an intellectual job.