java tcp server to many connections

Question

I wrote a simple TCP server to transfare some user Data to it and save it in an simple MySQL table. If i now run more than 2000 clients after each other it stops working. While running i get some IO error java.io.EOFException you may also see the misstake i made for that. But the most importand is that i get this

 IO error java.net.SocketException: Connection reset
    Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread
    at java.lang.Thread.start0(Native Method)
    at java.lang.Thread.start(Unknown Source)
    at Server.main(Server.java:49)

Enough Memory schould be there but the threads are still running and i dont see where i made the misstake that they dont get terminated. So i got up to 3900 threads running than. So here is the part of the Server:

try {
    // create new socket
    ServerSocket sock = new ServerSocket(port);
    textArea.setText(textArea.getText() + "Server started\n");
    while (true) {
        // accept the connection
            Socket newsock = sock.accept();
        // handle the action
        Thread t = new ThreadHandler(newsock);
            newsock.setSoTimeout(2000); // adding client timeout
        t.start();
        }
    } catch (Exception e) {

guess really simple. Here is how i handle the socket:

class ThreadHandler extends Thread {
    private Socket socket;
    private MySQLConnection sqlConnection;

    ThreadHandler(Socket s) {
        socket = s;
        sqlConnection = new MySQLConnection();
    }

    public void run() {
        try {
            DataOutputStream out = new DataOutputStream(
                    socket.getOutputStream());
            DataInputStream in = new DataInputStream(new BufferedInputStream(
                    socket.getInputStream()));
            Server.textArea.append((new Date()) + "\nClient connected IP: " + socket.getInetAddress().toString()+"\n");

            int firstLine = in.readInt(); // get first line for switch

            switch (firstLine) {
            case 0:
                // getting the whole objekt for the database in own lines!
                String name2 = in.readUTF();
                int level2 = in.readInt();
                int kp2 = in.readInt();
                String skill = in.readUTF();

                LeadboardElement element2 = new LeadboardElement();
                element2.setName(name2);
                element2.setLevel(level2);
                element2.setKillPoints(kp2);
                element2.setSkill(skill);
                sqlConnection.saveChaToLeadboard(element2);
                break;
                //case 1 return the top10
###.... shorten here the rest of the cases
                out.close();
            in.close();
            //close this socket
            socket.close();
                    Server.textArea.append("Client disconnected IP: " + socket.getInetAddress().toString()+ "\n" + (new Date())
                            + "\n----------------------------------------------------\n");
            // autoscrolldown
            Server.textArea.setCaretPosition(Server.textArea.getDocument()
                    .getLength());
         } catch (Exception e) {
            System.out.println("IO error " + e);
            try {
                socket.close();
            } catch (IOException e1) {
                e1.printStackTrace();
            }
        }finally{
        try {
            socket.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

the saveChaToLeadboard simply gets the name level kp and skill and uses a preparedStatement so save it to my MySQL Table. I hope you can help me i just dont see the misstake of it. I think i need to Join it somewhere but if i put a join at the end of it (after socket.close()) it still does the same.

Here the save to database methode:

public void saveChaToLeadboard(LeadboardElement element) {
        try {
            // load driver
            Class.forName("com.mysql.jdbc.Driver");
            connection = DriverManager.getConnection(this.databaseURL
                    + DATABASE_NAME, this.user, this.password);
            // insert values into the prep statement
            preparedStatement = connection
                    .prepareStatement(PREP_INSERT_STATEMENT);
            preparedStatement.setString(1, element.getName());
            preparedStatement.setInt(2, element.getLevel());
            preparedStatement.setInt(3, element.getKillPoints());
            if(!element.getSkill().equalsIgnoreCase("")){
                preparedStatement.setString(4, element.getSkill());
            }else{
                preparedStatement.setString(4, null);
            }
            // execute
            preparedStatement.executeUpdate();
            connection.close();

        } catch (Exception e) {
            Server.textArea.append(e.getMessage() + "\n");
            Server.textArea.setCaretPosition(Server.textArea.getDocument()
                    .getLength());
            try {
                connection.close();
            } catch (SQLException e1) {
                e1.printStackTrace();
            }
        }

Thanks alot! Regards

Answer 1

Your run() method is mangled, but I suspect that part of the problem is that you are not always closing network sockets and streams. In particular, I suspect that you are not closing them if there is an exception while reading, or processing the data you read. You should always close sockets and streams in a finally block (or the Java 7 equivalent).

Another potential problem is that some of the connections may be stalling due to the other end not sending data. To deal with that, you would need to set a read timeout on the socket ... so that connections to slow / stuck clients can be closed.

Finally, it is probably unrealistic to even try to process 2000+ connections in parallel with a thread per connection. That's a LOT of resources ¹ . I recommend you use a thread pool with a fixed upper limit in the low hundreds, and stop accepting new connections if all threads are in use.

---

^{1 - Each thread stack occupies at least 64K of memory on a HotSpot JVM, and possibly as much of 1Mb. Then there are the Heap resources that the thread directly or indirectly refers to, and OS resources needed to maintain the state of the threads and the sockets. For 2000 threads, that's probably multiple Gb of memory.}

Answer 2

IMHO 2000 threads is on the high side for a single process and 2000 database connections definately is.

Regardless of whether or not you're hitting limits with 2000 incoming connections, your approach simply will not scale.

To acheive scalability you need to look at using resource pools - this means:

• a pool of reader threads reading from the sockets queuing the data for processing.
• a pool of worker threads processing the data queued by the reader threads.
• a pool of database connections used by the worker threads - this connection pool could be adjusted so that each worker thread has it's own connection but the important thing is that you don't continually open and close database connections.

Look at the concurreny API for the thread pools and the NIO API for the IO.

This arrangement will allow you to tune your server to acheive the desired throughput.