[英]Wget Download for Multiple Domains and Files
I need to download a list of files from each of the domains listed in another file. 我需要从另一个文件中列出的每个域中下载文件列表。 I've tried many times but I still failed.
我已经尝试了很多次,但还是失败了。
The example list files to download (eg, file.txt): 示例列出了要下载的文件(例如,file.txt):
1.jpg
2.jpg
3.jpeg
4.bmp
5.gif
The example list of domains (eg, url.lst): 域的示例列表(例如url.lst):
google.com
google.co.in
google.com.br
The script: 剧本:
#!/bin/bash
# Create an array files that contains list of filenames
urls=`cat "url.lst"`
files=`cat "file.txt"`
for ((file in "${files[@]}" && url in "${urls[@]}")); do
wget "${url}${file}"
done
I want to get it so it generates and runs the following commands: 我想获取它,以便它生成并运行以下命令:
wget google.com/1.jpg
wget google.com/2.jpg
wget google.com/3.jpeg
wget google.com/4.bmp
wget google.com/5.gif
wget google.co.in/1.jpg
wget google.co.in/2.jpg
wget google.co.in/3.jpeg
wget google.co.in/4.bmp
wget google.co.in/5.gif
wget google.com.br/1.jpg
wget google.com.br/2.jpg
wget google.com.br/3.jpeg
wget google.com.br/4.bmp
wget google.com.br/5.gif
You have a few things going on here. 您正在这里进行几件事。 First, your reading the variables isn't creating arrays.
首先,您读取的变量不是在创建数组。 You're getting a string that will be subject to word splitting and globbing and the like.
您将得到一个字符串,该字符串将受到单词拆分和通配符等的限制。 Second, you will need to do the two file loops separately instead of trying to do it in a single command.
其次,您将需要分别执行两个文件循环,而不是尝试在单个命令中执行。
To fix the first part, I'd suggest using readarray
or mapfile
, for the second, use nested loops like: 要修复第一部分,我建议使用
readarray
或mapfile
,第二个建议使用嵌套循环,例如:
readarray -t urls < url.lst
readarray -t files < file.txt
for dom in "${urls[@]}"; do
for path in "${files[@]}"; do
wget "$dom/$path"
done
done
or you could replace the outer for
loop with a while
loop and skip one of the readarray
s like 或者您可以将外部
for
循环替换for
while
循环,并跳过其中的readarray
之一
readarray -t files < file.txt
while read -r url; do
for path in "${files[@]}"; do
wget "$url/$path"
done
done < url.lst
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.