Transfer a File over a network using TCP (Speed up the transfer)

Question

I have been trying to send a big file over a Socket connection, but it runs slowly and I was wondering if this code can be optimized in some way to improve the transfer speed.

This is my code for sending the file:

byte[] buffer = new byte[65536];
int number;

while ((number = fileInputStream.read(buffer)) != -1) {
    socketOutputStream.write(buffer, 0, number);
}

socketOutputStream.close();
fileInputStream.close();

This is what I use to receive the file on the other machine:

byte[] buffer = new byte[65536];

InputStream socketStream= clientSocket.getInputStream();
File f=new File("C:\\output.dat");

OutputStream fileStream=new FileOutputStream(f);

while ((number = socketStream.read(buffer)) != -1) {
    fileStream.write(buffer,0,number);
}

fileStream.close();
socketStream.close();

I think writing to the fileStream is taking the majority of the time. Could anyone offer any advise for speeding up this code.

Answer 1

There's nothing obviously wrong with that code, other than the lack of finally blocks for the close statements.

How long does it take for how much data? It's very unlikely that the FileOutputStream is what's taking the time - it's much more likely to be the network being slow. You could potentially read from the network and write to the file system in parallel, but that would be a lot of work to get right, and it's unlikely to give that much benefit, IMO.

Answer 2

You could try a BufferedOutputStream around the FileOutputStream. It would have the effect of block-aligning all disk writes, regardless of the count you read from the network. I wouldn't expect a major difference but it might help a bit.

Answer 3

I had a similar issue FTP'ing large files. I realized that using the same buffer for reading from the hard drive AND writing to the network was the issue. File system IO likes larger buffers because it is a lot less work for the hard drive to do all the seeking and reading. Networks on the other hand then to prefer smaller buffers for optimizing throughput.

The solution is to read from the hard disk using a large buffer, then write this buffer to the network stream in smaller chunks.

I was able to max out my NIC at 100% utilization for the entire length of any file with 4mb reads and 32kb writes. You can then do the mirrored version on the server by reading in 32kb at a time and storing it in memory then writing 4mb at a time to the hard drive.