How to get rid of tcp-ip send delay in interprocess communication between a java and a php process in KVM VM's

Question

I have a web application that consists of a java part and a php part. When a user does a request the php process will open a tcp/ip connection to the java process. It will keep this connection open for the duration of the request and this connection will be used to send a lot of information back and forth. This application runs very well as long as its hosted on either a dedicated server or on a VM that uses OpenVZ.

As soon as I try to host it on a KVM VM it becomes extremely slow. The reason for this is that within a single user request the php process can easily do up to 1 or 2 thousand tcp-ip sends to the java process. Now since this is all done over the same connection It really should not be a problem but on KVM VM's it seems each send gets about 20 milliseconds worth of delay so now a request that would normally take 0.1 seconds takes 20 seconds instead.

I'm not 100% sure KVM is to blame, But I have tested this on 3 different hosting provdiders using OpenVZ and another 3 different hosting providers using KVM. It runs perfectly fine on all the OpenVZ hosts and the send delay problem is present on all the KVM hosts.

O and I have tcpnodelay set on both the java and the php side.

Any idea what I could try to make this work on KVM?

Answer 1

So to answer my own question. It seems it seems you wont be able to avoid that send latency since even though its on localhost it still has to go from the virtualization layer down to the network layer and back up.

However, instead of creating TCP sockets on localhost the solution was to use Unix sockets instead. Since Unix sockets do not access the network layer in any way.

And as a bonus Using Unix sockets instead of TCP sockets gave my application a nice across the board performance boost. Including on setups were it worked fine before.