通过套接字发送大文件时出错

Question

I'm trying to send large files via socket. The program works fine for small files (such as html pages or pdf), but when i send files over 3/4 mb the output is always corrupted (viewing it with a text editor i noticed that the last few lines are always missing).

Here's the code of the server:

BufferedInputStream in = null;
    FileOutputStream fout = null;
    try {
        server = new ServerSocket(port);

        sock = server.accept();
        in = new BufferedInputStream(sock.getInputStream());

        setPerc(0);

        received = 0;

        int incByte = -1;
        fout = new FileOutputStream(path+name, true);
        long size = length;
        do{
            int buffSize;
            if(size >= 4096){
                buffSize = 4096;
            }else{
                buffSize = 1;
            }
            byte[] o = new byte[buffSize];
            incByte = in.read(o, 0, buffSize);
            fout.write(o);

            received+=buffSize;
            setPerc(calcPerc(received, length));
            size -= buffSize;
            //d("BYTE LETTI => "+incByte);
        }while(size > 0);
        server.close();
    } catch (IOException e) {
        e("Errore nella ricezione file: "+e);
    }finally{
        try {
            fout.flush();
            fout.close();
            in.close();
        } catch (IOException e) {
            e("ERRORE INCOMINGFILE");
        }
    }
    pr.release(port);

And here's the code of the client:

FileInputStream fin = null;
    BufferedOutputStream out = null;
    try {
        sock = new Socket(host, port);

        fin = new FileInputStream(file);
        out = new BufferedOutputStream(sock.getOutputStream());

        long size = file.length();
        int read = -1;
        do{
            int buffSize = 0;
            if(size >= 4096){
                buffSize = 4096;
            }else{
                buffSize = (int)size;
            }

            byte[] o = new byte[buffSize];
            for(int i = 0; i<o.length;i++){
                o[i] = (byte)0;
            }
            read = fin.read(o, 0, buffSize);
            out.write(o);
            size -= buffSize;
            //d("BYTE LETTI DAL FILE => "+read);
        }while(size > 0);
    } catch (UnknownHostException e) {
    } catch (IOException e) {
        d("ERRORE NELL'INVIO DEL FILE: "+e);
        e.printStackTrace();
    }finally{
        try {
            out.flush();
            out.close();
            fin.close();
        } catch (IOException e) {
            d("Errore nella chiusura dei socket invio");
        }
    }

i think it's something related with the buffer size, but i can't figure out what's wrong here.

Answer 1

This is incorrect:

        byte[] o = new byte[buffSize];
        incByte = in.read(o, 0, buffSize);
        fout.write(o);

You are reading up to buffSize bytes and then writing exactly buffSize bytes.

You are doing the same thing at the other end as well.

You may be able to get away with this when reading from a file ¹ , but when you read from a socket then a read is liable to give you a partially filled buffer, especially if the writing end can't always keep ahead of the reading end 'cos you are hammering the network with a large transfer.

The right way to do it is:

        incByte = in.read(o, 0, buffSize);
        fout.write(o, 0, incByte);

---

^{1 - It has been observed that when you read from a local file, a read call will typically give you all of the bytes that you requested (subject to the file size, etc). So, if you set buffSize to the length of the file, this code would probably work when reading from a local file. But doing this is a bad idea, because you are relying behaviour that is not guaranteed by either Java or a typical operating system.}

Answer 2

You might have a problem eg here.

read = fin.read(o, 0, buffSize);
out.write(o);

Here read gives you the count of bytes you've actually just read. On the next line you should write out only as many bytes as you've read.

In other words, you cannot expect the size of the file you're reading to be multiple of your buffer size.

Review your server code too for the same issue.

Answer 3

The correct way to copy streams in Java is as follows:

while ((count = in.read(buffer)) > 0)
{
    out.write(buffer, 0, count);
}

where count is an int, and buffer is a byte[] array of length > 0, typically 8k. You don't need to allocate byte arrays inside the loop, and you don't need a byte array of a specific size. Specifically, it's a complete waste of space to allocate a buffer as large as the file; it only works up to files of Integer.MAX_VALUE bytes, and it doesn't scale.

You do need to save the count returned by 'read()' and use it in the 'write()' method as shown above.