Java NIO read large file from inputstream

Question

I want to read a large InputStream and return it as a file. So I need to split InputStream(or I should read InputStream in multiple threads). How can I do this? I'm trying to do something like this:

    URL url = new URL("path");
    URLConnection connection = url.openConnection();
    int fileSize = connection.getContentLength();

    InputStream is = connection.getInputStream();
    ReadableByteChannel rbc1 = Channels.newChannel(is);
    ReadableByteChannel rbc2 = Channels.newChannel(is);

    FileOutputStream fos = new FileOutputStream("file.ext");

    FileChannel fileChannel1 = fos.getChannel();
    FileChannel fileChannel2 = fos.getChannel();
    fileChannel1.transferFrom(rbc1, 0, fileSize/2);
    fileChannel2.transferFrom(rbc2, fileSize/2, fileSize/2);

    fos.close();

But it does not affect on performance.

Answer 1

You can open multiple (HTTP) Connections to the same resource (URL) but use theRange: Header of HTTP to make each stream begin to read at another point. This can actually speed up the data transfer, especially when high latency is an issue. You should not overdo the parallelism, be aware that it puts additional load on the server.

connection1.setRequestProperty("Range", "bytes=0-" + half);
connection2.setRequestProperty("Range", "bytes=" + half+1 +"-");

This can also be used to resume downloads. It needs to be supported by the server. It can announce this with Accept-Ranges: bytes but does not have to . Be prepared that the first connection might return the whole requested entity (status 200 vs. 206) instead.

You need to read the input streams from the URLConnections in separate threads as this is blocking IO (not sure if the NIO wrapping helps here).

Answer 2

You can use position(long) method for each channel to start reading for.

Check this.

http://tutorials.jenkov.com/java-nio/file-channel.html#filechannel-position

Besides, if you want download a file partially,

Parallel Downloading

To download multiple parts of a file parallelly, we need to create multiple threads. Each thread is implemented similarly to the simple thread above, except that it needs to download only a part of the downloaded file. To do that, the HttpURLConnection or its super class URLConnection provides us method setRequestProperty to set the range of the bytes we want to download.

 // open Http connection to URL HttpURLConnection conn = (HttpURLConnection)mURL.openConnection(); // set the range of byte to download String byteRange = mStartByte + "-" + mEndByte; conn.setRequestProperty("Range", "bytes=" + byteRange); // connect to server conn.connect();

This would be helpful for you.

I found this answer here, you can check complete tutorial.

http://luugiathuy.com/2011/03/download-manager-java/