如何在java中为套接字连接创建软件负载均衡器

Question

I am planning to create a sofware load balancer that will sit in front of many socket servers in linux. The clients will connect to the load balancer . The load balancer will maintain a table of alternative ips and their ports. It will connect the client to the best available ip and port and disconnect itself from the client.Thus it will go out of the scene(will no longer be connected to the client).In this condition only connected devices will be the client and the new socket server but NOT the load balancer.

Example : Cleint ip 10.1.2.3 port 1234  
           load balancer Ip 10.1.2.4 port 1235
           list of socket servers   in Load Balancer: 
           A Ip 10.1.2.4 port 1236 
           B Ip 10.1.2.4 port 1237
           C  Ip 10.1.2.5 port 1238
    Now 
for the 1st request to the load balancer from client, the load balancer  will establish a connection between the client &  server A and disconnect itself from client.
     for the 2nd request to the load balancer from client, the load balancer  will establish a connection between the client &  server B and disconnect itself from client.
     for the 3rd request to the load balancer from client, the load balancer  will establish a connection between the client &  server C and disconnect itself from client.

Any Help on implementing this in Java is greatly appreciated.

Answer 1

This miniature design can be helpful for apps in small devices like mobiles and tabs. There can be a limit set by the server on how many round trip can a request be allowed from a particular device. Off course scaling of the online servers will help reduce the round trip counts .

Answer 2

I would use redis to store the lookup table. Each load-balancer server will lookup in the lookup table in redis for a connection to the most available / most prioritized server . This lookup will return a single integer which is an index of the server. Each app in the client will store the server ip with their respective indexes. so this lookup is very fast and less than 30 ms. At this point the connection will be faster. NO redirect is needed. A fall back is also provided in case there is concurrent connections and the qouta on the desired server is finished by the time the app tries to connect to the looked up server. In this case it will again look up the most available server , ie, start over recursively until it gets connected successfully or all resourced are finished and the connection-request is marked as dead end.

How about reserving the connection for few milliseconds, for each lookup ? After the expiry of the delay to connect for that lookup, the page-file will be made available to occupy for a new connection. This will decrease the recursive lookup but also block the connectivity . The delay should be sufficient for a connection to establish which may vary. On the other hand new connections will be blocked during this delay , which can result in a bad user experience. You need to trade-off between these two: decrease lookup and block connectivity vs never block connectivity and endure recursive lookup which is very fast.

Answer 3

The difficulty is disconnecting the device from server side and connecting it with the intended server.
There can be a work around : use redirect instead.

• Each server is a load balancer and service provider to the client devices
• Each server will always keep track of its open file limits and preserver safe margin from the max openfile limits
• This pool will be used to check if any redirect is needed.
• When the open file limit reaches the safe limit, any further connection request will return the next available server ip to the client device.
• The next nearest available server ip can be maintained in a lookup table in memory.
• The device will check if the returned value starts with the redirect ip it will re connect automatically to the received ip.Otherwise it will assume that the device got the service from any of the server successfully.

Thus we can avoid the open file limit and connection refused error .

Answer 4

I do not quite understand the requirement that the load balancer should disconnect from the client. If your sockets are in fact TCP connections, as they appear to be, I do not see how you could offload the connection to a client running somewhere else without some low level hackery. For example, look at ldirectord from linux virtual server . This allows fully offloading the connection.

For pure simplicity, I'd just use HAProxy . Does most of what you want, except for offloading the connection.

Finally, you could also use some kind of round-robin DNS solution. That would also offload the connection as you require.