Incomplete file using RandomAccessFile in java

Question

I made a small program to download data and write it to a file.

Here is the code:

public void run()
{

    byte[] bytes = new byte[1024];
    int bytes_read;

    URLConnection urlc = null;
    RandomAccessFile raf = null;
    InputStream i = null;


    try
    {
         raf = new RandomAccessFile("file1", "rw");
    }
    catch(Exception e)
    {
        e.printStackTrace();
        return;
    }

    try
    {
         urlc =  new URL(link).openConnection();
         i = urlc.getInputStream();
    }
    catch(Exception e)
    {
        e.printStackTrace();
        return;
    }

    while(canDownload())
    {
        try
        {
            bytes_read = i.read(bytes);
        }
        catch(Exception e)
        {
            e.printStackTrace();
            return;
        }

        if(bytes_read != -1)
        {
            try
            {
                raf.write(bytes, 0, bytes_read);
            }
            catch(Exception e)
            {
                e.printStackTrace();
                return;
            }
        }
        else
        {
            try
            {
                i.close();
                raf.close();
                return;
            }
            catch(Exception e)
            {
                e.printStackTrace();
                return;
            }
        }
    }
}

The problem is that when I download big files, I get few bytes missing in the end of the file. I tried to change the byte array size to 2K, and the problem was solved. But when I downloaded a bigger file (500 MB) , I got few bytes missing again. I said "Ok, let's try with 4K size". And I changed the byte array size to 4K. It worked! Nice, but then I downloaded a 4 GB file, I got bytes missing in the end again! I said "Cool, let's try with 8K size". And then I changed the byte array size to 8K. Worked.

My first question is: Why this happens? (when I change buffer size, the file doesn't get corrupted).

Ok, in theory, the file corrupted problem can be solved changing the byte array size to bigger values. But there's another problem: how can I measure the download speed (in one second interval) with big byte array sizes?

For example: Let's say that my download speed is 2 KB/s. And the byte array size is 4 K. My second question is: How can I measure the speed (in one second interval) if the thread will have to wait the byte array to be full? My answer should be: change the byte array size to a smaller value. But the file will get corrupted xD.

After trying to solve the problem by myself, I spent 2 days searching over the internet for a solution. And nothing.

Please, can you guys answer my two questions? Thanks =D

Edit

Code for canDownload():

synchronized private boolean canDownload()
{
    return can_download;
}

Answer 1

My advice is to use a proven library such as Apache Commons IO instead of trying to roll your own code. For your particular problem, take a look at the copyURLToFile(URL, File) method.

Answer 2

I would:

1. Change the RandomAccessFile to a FileOutputStream.

2. Get rid of canDownload() , whatever it's for, and set a read timeout on the connection instead.

3. Simplify the copy loop to this:

   while ((bytes_read = i.read(bytes)) > 0) { out.write(bytes, 0, bytes_read); } out.close(); i.close();

with all the exception handling outside this loop.

Answer 3

I think you will find the problem is that you closed the underlying InputStream while the RandomAccessFile still had data in its write buffers. This will be why you are occasionally missing the last few bytes of data.

The race condition is between the JVM flushing the final write, and your call to i.close().

Removing the i.close() should fix the problem; it isn't necessary as the raf.close() closes the underlying stream anyway, but this way you give the RAF a chance to flush any outstanding buffers before it does so.