Java nio 仅读取 8192/433000 字节

Question

I have a project that I'm working on to better understand Java NIO and network programming stuff. I'm trying to send a 400,000+ byte file through netcat to my server where it will be found and written to a file.

The Problem: The program works perfectly when the file is below 10,000 bytes, or when I place a Thread.sleep(timeout) before the Select(). The file sends over but only reads 8192 bytes and then cancels out of the loop and goes back to the select() to capture the rest of the data. However the file captures what comes after. I need the complete data for further expansion to the project.

Things I've Tried: I've tried to load the data onto another byte array which evidently works, but skips over the 8192 bytes (since the select() has been called again). Reads the rest of the 391000 bytes. When comparing the files the first 8192 bytes is missing. I've tried various other things but I'm not adequate in NIO to understand what I'm messing up on.

My Code

This is just where I feel the code is messing bout (after debugging)

private void startServer() {
   File temp = new File("Filepath");
   Selector selector = Selector.open();
   ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
   serverSocketChannel.configureBlocking(false);
   serverSocketChannel.socket().bind(listenAddress);
   serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);

   log.info("Server Socket Channel Started");

   while(!stopRequested){
     selector.select();
     Set<SelectionKey> keys = selector.selectedKeys();

     for(SelectionKey key : keys){
       if(key.isAcceptable()){
         try {
           serverSocketChannel = (ServerSocketChannel) key.channel();
           SocketChannel socket = serverSocketChannel.accept();
           socket.configureBlocking(false);
           socket.register(selector, SelectionKey.OP_READ);
         }catch (IOException e) {
           log.error("IOException caught: ", e);
         }
       }
       if(key.isReadable(){ read(key); }
       keys.remove(key);
     }
   } 
  } catch (IOException e) {
    log.error("Error: ", e);
  }
}

private void read(SelectionKey key) {
  int count = 0;
  File tmp = new File("Path");

  try {
    SocketChannel channel = (SocketChannel) key.channel();
    byteBuffer.clear();
    while((count = channel.read(byteBuffer)) > 0) {
         byteBuffer.flip();
         //in bytearrayoutputstream to append to data byte array
         byteArrayOutputStream.write(byteBuffer.array(), byteBuffer.arrayOffset(), count);
         byteBuffer.compact();
      }
    }
    data = byteArrayOutputStream.toByteArray();
    FileUtils.writeByteArrayToFile(tmp, data);
  }
}

The above code is what I'm working with. I have more stuff in this class but I believe the main two functions having the problem are these two. I'm not too sure what steps I should take. The file I have to test my program contains many TCPs about 400,000 bytes. The select() collects the initial 8192 bytes and then runs read (which shouldn't happen until it captures all of the data in the stream), comes back and gathers the rest. I've allocated the byteBuffer to be 30720 bytes.

If not clear, I can post the rest of the code, let me know what your suggestions are.

Question
Why does this code only grab 8192 bytes when the allocated space is 30720? Why does it work in debug mode or with Thread.sleep()?

Previous person advised me to place my byteBuffer.clear() outside of loop, even after doing so, the problem persists.

Answer 1

The non-blocking API merely promises that the 'readable' state is raised if there are more than 0 bytes. It makes no guarantee that it'll wait until all the bytes you're interested in have arrived; there isn't even a way to say 'dont mark this channel as isReadable until at least X bytes are in'. There is no way to fix that directly; your code must instead be capable of dealing with a half filled buffer. For example, by either reading this data away so that the 'isReadable' state gets cleared until MORE bytes arrive.

Using the raw non-blocking APIs is rocket science (as in, it is very tricky to write your code correctly, it is easy to get a CPU core to spin to 100% because you're mismanaging your flags, and it is easy to have all threads frozen and the app reduced to being able to handle only a percent or two of what a normal threaded variant could have done due to accidental invocation of blocking methods.

I strongly suggest you first reconsider whether you need non-blocking at all (it always almost slower, and orders of magnitude harder to develop for. After all, you cannot make a single potentially blocking call anywhere in any handler code or your app will be dog slow under load, and java is not great at await/yield stuff – the only real benefit is that you get more finegrained control over buffer sizes, which is irrelevant unless you are very RAM constrained and can get away with tiny buffers for per-connection state). And if you then conclude that truly this is the only way, use a library that makes this API easier to use, such as netty .