FileInputStream和DataOutputStream - 处理byte []缓冲区

Question

I've been working on an app to move files between two hosts and while I got the transfer process to work (code is still really messy so sorry for that, I'm still fixing it) I'm kinda left wondering how exactly it handles the buffer. I'm fairly new to networking in java so I just don't want to end up with "meh i got it to work so let's move on" attitude.

File sending code.

    public void sendFile(String filepath, DataOutputStream dos) throws Exception{
    if (new File(filepath).isFile()&&dos!=null){
        long size = new File(filepath).length();
        String strsize = Long.toString(size) +"\n";
        //System.out.println("File size in bytes: " + strsize);
        outToClient.writeBytes(strsize);
        FileInputStream fis = new FileInputStream(filepath);
        byte[] filebuffer = new byte[8192];

        while(fis.read(filebuffer) > 0){
            dos.write(filebuffer);
            dos.flush();
        }

File recieving code

   public void saveFile() throws Exception{
    String size = inFromServer.readLine();
    long longsize = Long.parseLong(size);
    //System.out.println(longsize);
    String tmppath = currentpath + "\\" + tmpdownloadname;
    DataInputStream dis = new DataInputStream(clientSocket.getInputStream());
    FileOutputStream fos = new FileOutputStream(tmppath);
    byte[] filebuffer = new byte[8192];
    int read = 0;
    int remaining = (int)longsize;
    while((read = dis.read(filebuffer, 0, Math.min(filebuffer.length, remaining))) > 0){
        //System.out.println(Math.min(filebuffer.length, remaining));
        //System.out.println(read);
        //System.out.println(remaining);
        remaining -= read;
        fos.write(filebuffer,0, read);
    }

}

I'd like to know how exactly buffers on both sides are handled to avoid writing wrong bytes. (ik how receiving code avoids that but i'd still like to know how byte array is handled)

Does fis/dis always wait for buffers to fill up fully? In receiving code it always writes full array or remaining length if it's less than filebuffer.length but what about fis from sending code.

Answer 1

In fact, your code could have a subtle bug, exactly because of the way you handle buffers.

When you read a buffer from the original file, the read(byte[]) method returns the number of bytes actually read. There is no guarantee that, in fact, all 8192 bytes have been read.

Suppose you have a file with 10000 bytes. Your first read operation reads 8192 bytes. Your second read operation, however, will only read 1808 bytes. The third operation will return -1.

In the first read, you write exactly the bytes that you have read, because you read a full buffer. But in the second read, your buffer actually contains 1808 correct bytes, and the remaining 6384 bytes are wrong - they are still there from the previous read.

In this case you are lucky, because this only happens in the last buffer that you write. Thus, the fact that you stop reading on your client side when you reach the pre-sent length causes you to skip those 6384 wrong bytes which you shouldn't have sent anyway.

But in fact, there is no actual guarantee that reading from the file will return 8192 bytes even if the end was not reached yet. The method's contract does not guarantee that, and it's up to the OS and underlying file system. It could, for example, send you 5000 bytes in your first read, and 5000 in your second read. In this case, you would be sending 3192 wrong bytes in the middle of the file.

Therefore, your code should actually look like:

byte[] filebuffer = new byte[8192];
int read = 0;
while(( read = fis.read(filebuffer)) > 0){
    dos.write(filebuffer,0,read);
    dos.flush();
}

much like the code you have on the receiving side. This guarantees that only the actual bytes read will be written.

So there is nothing actually magical about the way buffers are handled. You give the stream a buffer, you tell it how much of the buffer it's allowed to fill, but there is no guarantee it will fill all of it. It may fill less and you have to take care and use only the portion it tells you it fills.

Another grave mistake you are making, though, is to just convert the long that you received into an int in this line:

int remaining = (int)longsize;

Files may be longer than an integer contains. Especially things like long videos etc. This is why you get that number as a long in the first place. Don't truncate it like that. Keep the remaining as long and change it to int only after you have taken the minimum (because you know the minimum will always be in the range of an int ).

long remaining = longsize;
long fileBufferLen = filebuffer.length;

while((read = dis.read(filebuffer, 0, (int)Math.min(fileBufferLen, remaining))) > 0){
    ...
}

By the way, there is no real reason to use a DataOutputStream and DataInputStream for this. The read(byte[]) , read(byte[],int,int) , write(byte[]) , and write(byte[],int,int) are inherited from the underlying InputStream and there is no reason not to use the socket's OutputStream / InputStream directly, or use a BufferedOutputStream / BufferedOutputStream to wrap it. There is also no need to use flush until you have finished writing/reading.

Also, do not forget to close at least your file input/output streams when you are done with them. You may want to keep the socket input/output streams open for continued communication, but there is no need to keep the files themselves open, it may cause problems. Use a try-with-resources to guarantee that they are closed.