如何使用递归操作从不同的 URL 下载多个文件？

Question

I used ExecutorService for downloading files from different URLs concurrently but the time it takes is not small compared to sequential downloading; therefore, I want to use RecursiveAction. Below is the code to be applied:

Parallel class:

String[] links;
File[] files;

public Parallel(String[] link, File[] files) {
    this.links = link;
    this.files = files;
}

@Override
public void run() {
    try {
        for (int i = 0; i <= 3; i++) {
            URL url = new URL(links[i]);
            HttpURLConnection http = (HttpURLConnection) url.openConnection();
            double fileSize = (double) http.getContentLengthLong();
            BufferedInputStream bis = new BufferedInputStream(http.getInputStream());
            FileOutputStream fos = new FileOutputStream(this.files[i]);
            BufferedOutputStream bos = new BufferedOutputStream(fos, 600000);
            byte[] buffer = new byte[1024];
            double downloadedData = 0.00;
            int readData = 0;

            while ((readData = bis.read(buffer, 0, 1024)) >= 0) {
                bos.write(buffer, 0, readData);
                downloadedData += readData;
            }
            bos.close();
            bis.close();
            System.out.println(this.files[i] + " -> done");
        }
    } catch (IOException ex) {
        ex.printStackTrace();
    }
}

In main class, I just give the links and paths to files and start the execution

ExecutorService executor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
        Runnable worker = new Parallel(links, files);
        executor.execute(worker);
        executor.shutdown();

Any help is appreciated!

Answer 1

You are not using concurrency properly here.

What you should do is something like that:

String link;
File file;

public Parallel(String link, File file) {
    this.link = link;
    this.file = files;
}

@Override
public void run() {
    try {
        URL url = new URL(link);
        HttpURLConnection http = (HttpURLConnection) url.openConnection();
        double fileSize = (double) http.getContentLengthLong();
        BufferedInputStream bis = new BufferedInputStream(http.getInputStream());
        FileOutputStream fos = new FileOutputStream(file);
        BufferedOutputStream bos = new BufferedOutputStream(fos, 600000);
        byte[] buffer = new byte[1024];
        double downloadedData = 0.00;
        int readData = 0;

        while ((readData = bis.read(buffer, 0, 1024)) >= 0) {
            bos.write(buffer, 0, readData);
            downloadedData += readData;
        }
        bos.close();
        bis.close();
        System.out.println(file + " -> done");
    } catch (IOException ex) {
        ex.printStackTrace();
    }
}

And then:

String[] links;
File[] files;

//...

ExecutorService executor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());

for (int i = 0; i <= 3; i++) {
    Runnable worker = new Parallel(links[i], files[i]);
    executor.execute(worker);
}
executor.shutdown();

Then each download would actually get its own thread.

In your case all downloads get one thread where it all happens sequentally.