TCP文件传输中缺少字节

Question

I need to be able to read a file, break it into packets of some arbitrary size, lets say 512 bytes, and send those packets over TCP. Problem is, the receiver isn't getting all the bytes that im sending. If I send 1000 packets, the receiver blocks when reading from InputStream because he has no more data to read around 990 packets or so.

Here's the code (just the sending and receiving parts):

Sender:

int parts = (int)Math.ceil((double)(file.length()/512.0));

out.println(parts+"");
int readFile;
int i = 0;
while ((readFile = fileIn.read(buffer)) != -1) {
   i++;
   fileOut.write(buffer, 0, readFile);
   fileOut.flush();
   System.out.println("-- Sent packet " + i + "/" + parts + ". " + "Bytes sent = " + readFile);
}

Receiver:

int parts = Integer.parseInt(in.readLine());
byte[] buffer = new byte[512];
FileOutputStream pw = new FileOutputStream("file.ext");
DataInputStream fileIn = new DataInputStream(socket.getInputStream());
for(int j = 0; j < parts; j++){
   int read = 0;
   if(j == parts - 1){
      read = fileIn.read(buffer);
      pw.write(buffer, 0, read);
   }else{
      fileIn.readFully(buffer);
      pw.write(buffer);
   }
   System.out.println("-- Received packet " + (j+1) + "/" + parts + ". Read " +read+ " bytes.");
}

I tried increasing the socket's send and receive buffer size but without success. What am I missing?

Here is an output sample:

Sender:

-- Sent packet 1/10. Bytes sent = 512
-- Sent packet 2/10. Bytes sent = 512
-- Sent packet 3/10. Bytes sent = 512
-- Sent packet 4/10. Bytes sent = 512
-- Sent packet 5/10. Bytes sent = 512
-- Sent packet 6/10. Bytes sent = 512
-- Sent packet 7/10. Bytes sent = 512
-- Sent packet 8/10. Bytes sent = 512
-- Sent packet 9/10. Bytes sent = 512
-- Sent packet 10/10. Bytes sent = 234

Receiver:

-- Received packet 1/10. Read 512 bytes.
-- Received packet 2/10. Read 512 bytes.
-- Received packet 3/10. Read 512 bytes.
-- Received packet 4/10. Read 512 bytes.
-- Received packet 5/10. Read 512 bytes.
-- Received packet 6/10. Read 512 bytes.
-- Received packet 7/10. Read 512 bytes. (And it blocks here, because there is no more data to read)

Answer 1

TCP is a stream protocol . There are no packets, just a single stream of data.

You shouldn't be making the assumption that a single write() (with or without a flush() ) will correspond to a single read() . Therefore your receive loop for(int j = 0; j < parts; j++) is misguided: a better approach would be to count the number of bytes read an be prepared for read() calls returning varying amounts of data.

In the comments you argue that readFully() takes care of the problem. However, my concern lies not so much with the code as with the packet-based view of the stream. This leads to bugs like that in your final fileIn.read(buffer) call. It might return half the data that got sent as part of your last "packet", and you'll never know!

Answer 2

break it into packets of some arbitrary size

Why? It's a streaming protocol. You can break it up as much as you like at the sending end, but the local TCP will then do its best to combine your writes into larger TCP segments; the local IP will then chunk that into MTU-sized IP packets; intermediate routers may fragment that further; the remote IP will then reassemble the fragments into packets; the remote TCP will then reassemble the packets into segments; and the remote application will then receive in chunks dependent on all that as well as on the size of its socket receive buffer in the kernel, and on the size of its application receive buffer.

Don't attempt to out-think all that processing. You can't. Just write what you have to write when you have it to write.

If you have 'missing bytes' it can only be because you are ignoring the length value returned by read() at the receiver, or if you are using non-blocking I/O at the sender, the length value returned by write() at the sender.