如何使用递归操作从不同的 URL 下载多个文件？

Question

我使用 ExecutorService 同时从不同的 URL 下载文件，但与顺序下载相比，它花费的时间并不短； 因此，我想使用 RecursiveAction。 下面是要应用的代码：

平行类：

String[] links;
File[] files;

public Parallel(String[] link, File[] files) {
    this.links = link;
    this.files = files;
}

@Override
public void run() {
    try {
        for (int i = 0; i <= 3; i++) {
            URL url = new URL(links[i]);
            HttpURLConnection http = (HttpURLConnection) url.openConnection();
            double fileSize = (double) http.getContentLengthLong();
            BufferedInputStream bis = new BufferedInputStream(http.getInputStream());
            FileOutputStream fos = new FileOutputStream(this.files[i]);
            BufferedOutputStream bos = new BufferedOutputStream(fos, 600000);
            byte[] buffer = new byte[1024];
            double downloadedData = 0.00;
            int readData = 0;

            while ((readData = bis.read(buffer, 0, 1024)) >= 0) {
                bos.write(buffer, 0, readData);
                downloadedData += readData;
            }
            bos.close();
            bis.close();
            System.out.println(this.files[i] + " -> done");
        }
    } catch (IOException ex) {
        ex.printStackTrace();
    }
}

在主类中，我只提供文件的链接和路径并开始执行

ExecutorService executor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
        Runnable worker = new Parallel(links, files);
        executor.execute(worker);
        executor.shutdown();

任何帮助表示赞赏！

Answer 1

您在这里没有正确使用并发。

你应该做的是这样的：

String link;
File file;

public Parallel(String link, File file) {
    this.link = link;
    this.file = files;
}

@Override
public void run() {
    try {
        URL url = new URL(link);
        HttpURLConnection http = (HttpURLConnection) url.openConnection();
        double fileSize = (double) http.getContentLengthLong();
        BufferedInputStream bis = new BufferedInputStream(http.getInputStream());
        FileOutputStream fos = new FileOutputStream(file);
        BufferedOutputStream bos = new BufferedOutputStream(fos, 600000);
        byte[] buffer = new byte[1024];
        double downloadedData = 0.00;
        int readData = 0;

        while ((readData = bis.read(buffer, 0, 1024)) >= 0) {
            bos.write(buffer, 0, readData);
            downloadedData += readData;
        }
        bos.close();
        bis.close();
        System.out.println(file + " -> done");
    } catch (IOException ex) {
        ex.printStackTrace();
    }
}

进而：

String[] links;
File[] files;

//...

ExecutorService executor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());

for (int i = 0; i <= 3; i++) {
    Runnable worker = new Parallel(links[i], files[i]);
    executor.execute(worker);
}
executor.shutdown();

然后每次下载实际上都会有自己的线程。

在您的情况下，所有下载都会得到一个线程，所有这些都依次发生。