使用 python + 请求从链接下载多个 zip 文件

Question

I am trying to download zipped files from the US census bureau ( https://www2.census.gov/geo/tiger/TIGER2019/PLACE/ ). The code I have so far appeared to work, but all of the files downloaded are empty. Can someone help fill in what i'm missing?

from bs4 import BeautifulSoup as bs
import requests
import re

DOMAIN = "https://www2.census.gov/"
URL = "https://www2.census.gov/geo/tiger/TIGER2019/PLACE/"


def get_soup(URL):
    return bs(requests.get(URL).text, 'html.parser')

for link in get_soup(URL).findAll("a", attrs={'href': re.compile(".zip")}):
    file_link = link.get('href')
    print(file_link)

with open(link.text, 'wb') as file:
    response = requests.get(DOMAIN + file_link)
    file.write(response.content)

Answer 1

It looks like you're using the wrong links.

After looking at the website I can see that the download links have the following URL: "https://www2.census.gov/geo/tiger/TIGER2019/PLACE/[file_link]",

So currently your DOMAIN variable is wrong it should instead be using your URL variable.

Currently, the "with open" also isn't indented to work with all the links in the for-loop so only the last link is downloaded in the current state your code.