read multiple streams from a stream of a socket using java?

Question

Is there any way to read bytes randomly from a open socket stream? suppose i have opened a data stream in a website. traditional java classes giving me sequential access to the data. like a linked list. i want to randomly read data like reading data from random positions of an array. Suppose if the data size is 2048 KB, i want to read from 0 to 1204 KB with 1 thread and simultaneously want to use another thread to read data from 1205 to 2048 KB.

so the bottom line is i want to multi thread the data reading process from a opened socket of a website. The process has to be like Internet Download Manager. Please give me some tutorials link if its possible..

Answer 1

Reading from random locations in a stream from a website using multiple threads is not possible. The data comes down as a linear stream and there is no standard way to go to different positions in the stream in multiple threads. Multiple threads can download different files a the same time however.

The HTTPD protocol does support the Range: header although I'm not sure how many web servers support the behavior -- Apache does for example. If supported, the server should respond with the following header:

Accept-Ranges: bytes

The client then can ask for a particular offset/length of the file:

Range: bytes=21010-47021

With this you could have multiple threads downloading from different ranges at the same time but I'm not sure how this will speed up the download unless you are requesting from multiple servers. Most likely you will be running into disk and network limitations.

Answer 2

The data from a socket arrives sequentially. You'd have to use some fairly large buffering to avoid this.

Answer 3

I thinkt what you are asking about is the http header

Accept-Ranges: bytes
Range: bytes=0-8999

Which will instruct the server to only send some of the file. You will then read the stream sequentially.

See also How to assemble the file using the range header?

edit: example

This seems to work

public static void main(String[] args) throws MalformedURLException, IOException {
    URLConnection conn = new URL("http://ftp.debian.org/debian/dists/stable/Contents-i386.gz")
            .openConnection();
    conn.addRequestProperty("Accept-Ranges","bytes");
    conn.addRequestProperty("Range", "bytes=8000000-16000000");
    InputStream input = conn.getInputStream();
    List<String> serverranges = conn.getHeaderFields().get("Accept-Ranges");
    boolean ispartial = serverranges != null && serverranges.get(0).equals("bytes");
    byte[] b = new byte[1024];
    int l ;
    System.out.println(ispartial);
    while((l=input.read(b, 0, b.length))>0){
        // if isPartial=true, we have server support. We received partial file.
        //do stuff with b,l
    }
}

It is important to note that not all servers support this, so check the isPartial variable. If it is false. The server does not support partial ranges and will give you the start of the file.

Answer 4

You cannot read randomly over a socket. In principle, you have two options:

1. Read the whole data stream and put it an a buffer (a byte array for instance). Expose the buffer to each thread.

2. Read the whole data stream in each thread independently, having each thread ignore everything that should not read.