Maximum size of data that can be fetched from a client socket using socketChannel of nio package in java

Question

Using SocketChannel sc =(SocketChannel)key.channel(); , we can fetch data from port into buffer.
In order to receive the data continuously from port without the loss of data, how the code should be ?

Here is my code

import java.io.*;
import java.net.*;
import java.nio.*;
import java.nio.channels.*;
import java.util.*;

public class MultiPortEcho
{
  private int ports[];
  private ByteBuffer echoBuffer = ByteBuffer.allocate(32000);

  public MultiPortEcho( int ports[] ) throws IOException
  {
    this.ports = ports;
    go();
  }

  private void go() throws IOException
  {
    // Create a new selector
    Selector selector = Selector.open();

    // Open a listener on each port, and register each one
    // with the selector
    for (int i=0; i<ports.length; ++i)
    {
      ServerSocketChannel ssc = ServerSocketChannel.open();
      ssc.configureBlocking( false );
      ServerSocket ss = ssc.socket();
      InetSocketAddress address = new InetSocketAddress( ports[i] );
      ss.bind( address );

      SelectionKey key = ssc.register( selector, SelectionKey.OP_ACCEPT );

      System.out.println( "Going to listen on "+ports[i] );
    }

    while (true)
    {
      int num = selector.select();
      System.out.println("num::::"+num);
      Set selectedKeys = selector.selectedKeys();
      Iterator it = selectedKeys.iterator();

      while (it.hasNext())
      {
        SelectionKey key = (SelectionKey)it.next();

        if ((key.readyOps() & SelectionKey.OP_ACCEPT)== SelectionKey.OP_ACCEPT)
          {
              // Accept the new connection
              ServerSocketChannel ssc = (ServerSocketChannel)key.channel();
              SocketChannel sc = ssc.accept();
              sc.configureBlocking( false );

              // Add the new connection to the selector
              SelectionKey newKey = sc.register(selector,SelectionKey.OP_READ);
              it.remove();

              System.out.println( "Got connection from "+sc );
        }
          else if ((key.readyOps() & SelectionKey.OP_READ)== SelectionKey.OP_READ)
          {
              // Read the data
              SocketChannel sc =(SocketChannel)key.channel();
              System.out.println("sc::::"+sc);
              // data to fetched from channel and dump into the datatbase
              int bytesEchoed = 0;
              //while(true)
              {
                    echoBuffer.clear();
                    int r = sc.read(echoBuffer);
                    System.out.println("r:::" + r);
                    /*int pos=echoBuffer.position();
                    System.out.println("pos:::" +pos);*/
                    if (r == -1)
                    {
                        //echoBuffer.flip();
                        echoBuffer.rewind();
                        byte[] array = new byte[100000];
                        while (echoBuffer.hasRemaining())
                        {
                            int n = echoBuffer.remaining();
                            System.out.println("size:" + n);
                            echoBuffer.get(array,0,n );
                            System.out.println(new String(array,0,n));
                            key.cancel();
                            it.remove();

                        }

                    }

                    /*int pos=echoBuffer.position();
                    System.out.println("pos:::" + pos);
                    if(r<=0)
                    {
                        echoBuffer.flip();
                        for (int j = 0; j < pos; j++ )
                        {
                            String ss =Integer.toHexString(echoBuffer.get());
                            if (ss.length() == 1)
                                System.out.print("0" + ss + " ");
                            else if (ss.length() > 2)
                                System.out.print(ss.substring(6) + " ");
                            else System.out.print(ss + " ");
                        }
                      break;
                    }

                    echoBuffer.flip();

                    sc.write( echoBuffer );
                    bytesEchoed += r;*/
              }

             //System.out.println( "Echoed "+bytesEchoed+" from "+sc );
             //it.remove();
        }

      }

//System.out.println( "going to clear" );
//      selectedKeys.clear();
//System.out.println( "cleared" );
    }
  }

  static public void main( String args[] ) throws Exception
  {
    FileOutputStream fileoutputstream = new FileOutputStream("MultiPort.txt", false);
    PrintStream printstream = new PrintStream(fileoutputstream);
    System.setOut(printstream);
    if (args.length<=0) {
      System.err.println( "Usage: java MultiPortEcho port [port port ...]" );
      System.exit( 1 );
    }

    int ports[] = new int[args.length];

    for (int i=0; i<args.length; ++i) {
      ports[i] = Integer.parseInt( args[i] );
    }

    new MultiPortEcho( ports );
  }
}

Answer 1

The maximum size you can read is effectively limited by the amount of memory you have.

However you don't need to read super large blocks for efficiency. You should find that 1 MB is more than enough. In fact you may find that blocks of 4KB are large enough to get maximum bandwidth for a 1 Gb connection.

Answer 2

A comment on the general design:

There are two basic ways to write network servers. Blocking and Nonblocking. In 2008, we had the task of implementing a high performance network server in Python. After trying a couple different ways with non-blocking, we found that it was much easier and more clear to use:

• blocking sockets
• one thread per connection
• a couple of manager threads

That way each thread could sit and wait on data until the day it died, and when it received a full packet, it would act on that.

Just for consideration.

Answer 3

You can find some leads in this SocketChannelHandler , where are readFromChannel() function might be of interest for you.

        public void readFromChannel() {
        try {
                   [...]
                   if (readBuffer != null) {
                readBuffer.flip();
                receivingBroker.broker(readBuffer, false);
                if (readBuffer != null) {
                    readBuffer.clear();
                    readBuffer = null;
                }
            }
            if (readBuffer == null || !readBuffer.hasRemaining()) {
                getThread().removeInterestOp(this, SelectionKey.OP_READ);
                getThread().addInterestOp(this, SelectionKey.OP_WRITE);
            }
            if (receivingBroker.isClosed()) {
                if (getChannelListener() != null) {
                    getChannelListener().readFinished(this);
                }
            }
         } catch (Exception e) {
              e.printStackTrace();
         }
         }

Answer 4

As a first fix you should remove the line with key.cancel(). Keeping it will cancel the key and insure the key isn't considered after the first read - which will effectively stop you from reading anything afterwards.

Answer 5

When you accept a connection using NIO you can get hold of the socket and set the respective in / out buffer sizes.

socketChannel.socket().setReceiveBufferSize(512);
socketChannel.socket().setSendBufferSize(16);

Since NIO makes heavy use of the OS's network stack this is just a hint. All this is actually very well documented in the Socket JavaDoc