如何使用Java下载远程文件

Question

I'm trying to download a single file from a web server (http or https) using as few third party libraries as possible.

The method I've come up with is as follows:

private static final int BUFFER_SIZE = 8;

public static boolean download(URL url, File f) throws IOException {
    URLConnection conn = url.openConnection();
    conn.setDoOutput(true);

    FileOutputStream out = new FileOutputStream(f);
    BufferedInputStream in = new BufferedInputStream(conn.getInputStream());

    byte[] buffer;
    long dld = 0, expected = conn.getContentLengthLong(); // TODO expected will be -1 if the content length is unknown
    while (true) { // TODO fix endless loop if server timeout
        buffer = new byte[BUFFER_SIZE];
        int n = in.read(buffer);
        if (n == -1) break;
        else dld += n;
        out.write(buffer);
    }
    out.close();
    System.out.println(dld + "B transmitted to " + f.getAbsolutePath());
    return true;
}

However, it does by no means work as intended. I tried to download https://upload.wikimedia.org/wikipedia/commons/6/6d/Rubber_Duck_Florentijn_Hofman_Hong_Kong_2013d.jpg for example, the result was horrifying:

For some reason I was able to view the picture in IrfanView but not in any other viewer, so this is a re saved version.

I tried messing with the buffer size or downloading other images but the results are more or less the same.

If I look at the file, there are entire parts of the content simply replaced with dots:

I'm really lost on this one so thanks for any help :)

Answer 1

The problem occurs when there aren't 8 bytes of data to read. This leaves part of the array filled with zeros, which is why you're seeing so many in your hex editor.

The solution is simple: replace out.write(buffer); with out.write(buffer, 0, n); . This tells the FileOutputStream to only read the bytes between indexes 0 and n .

Fixed code:

private static final int BUFFER_SIZE = 8;

public static boolean download(URL url, File f) throws IOException {
    URLConnection conn = url.openConnection();
    conn.setDoOutput(true);

    FileOutputStream out = new FileOutputStream(f);
    BufferedInputStream in = new BufferedInputStream(conn.getInputStream());

    // We can move the buffer declaration outside the loop
    byte[] buffer = new byte[BUFFER_SIZE];

    long dld = 0, expected = conn.getContentLengthLong(); // TODO expected will be -1 if the content length is unknown
    while (true) {
        int n = in.read(buffer);
        if (n == -1) break;
        else dld += n;
        out.write(buffer, 0, n);
    }
    out.close();
    System.out.println(dld + "B transmitted to " + f.getAbsolutePath());
    return true;
}

Answer 2

Try something like this to download pictures

public static byte[] download(String param) throws IOException {
    InputStream in = null;
    ByteArrayOutputStream out = null;

    try {
        URL url = new URL(param);
        HttpURLConnection con = (HttpURLConnection)url.openConnection(); 
        con.setConnectTimeout(120000);
        con.setReadTimeout(120000);
        con.setRequestMethod("GET");
        con.connect();
        in = new BufferedInputStream(con.getInputStream());
        out = new ByteArrayOutputStream();
        byte[] buf = new byte[1024];
        int n = 0;
        while (-1 != (n = in.read(buf))) {
            out.write(buf, 0, n);
        }

        return out.toByteArray();

    } finally {
        try {
            out.close();
        } catch (Exception e1) {

        }
        try {
            in.close();
        } catch (Exception e2) {

        }
    }
}