使用 Selenium 和 Beautiful Soup 點擊“下載 csv”按鈕

Question

我正在嘗試從該網站下載 csv 文件： https://invasions.si.edu/nbicdb/arrivals?state=AL&submit=Search+database&begin=2000-01-01&end=2020-General&bwms- =任何

為此，我需要單擊 CSV 按鈕，該按鈕會下載 CSV 文件。 但是，我需要對多個鏈接執行此操作，這就是為什么我想使用 Selenium 來自動執行單擊鏈接的任務。

我當前運行的代碼，但實際上並沒有將 csv 文件下載到指定文件夾（或任何地方）。

這是我目前擁有的代碼：

import selenium
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
import time

options = webdriver.ChromeOptions() 
options.add_argument("download.default_directory=folder") # Set the download Path
driver = webdriver.Chrome(options=options)

url = 'https://invasions.si.edu/nbicdb/arrivals?state=AL&submit=Search+database&begin=2000-01-01&end=2020-11-11&type=General+Cargo&bwms=any'

driver.get(url)

python_button = driver.find_element_by_xpath('//*[contains(concat( " ", @class, " " ), concat( " ", "csvbutton", " " ))]')
python_button.click()

我將不勝感激任何幫助！ 謝謝

Answer 1

您可以使用這種方式解決您的問題：

import requests
from bs4 import BeautifulSoup

# url of initial page with data
url = 'https://invasions.si.edu/nbicdb/arrivals?state=AL&submit=Search+database&begin=2000-01-01&end=2020-11-11&type=General+Cargo&bwms=any'
# name of csv file where to store downloaded csv data
csv_file_name = '/Users/eilyasov/Documents/arrivals_data.csv'

# get html content of initial page
html_data = requests.get(url=url) \
                    .content
# generate Beautifulsoup object based on hrml content of initial page
soup = BeautifulSoup(markup=html_data)
# extract url extension of downloadable csv file
csv_url_extension = soup.find(name='a', attrs={'class': 'csvbutton'}) \
                        .get(key='href')
# construct url of downloadable csv file
csv_url = 'https://invasions.si.edu' + csv_url_extension
# get content of downloadable csv file and saving it to file
response = requests.get(url=csv_url)
if response.status_code == 200:
    with open(csv_file_name, 'wb') as file:
        file.write(response.content)