使用 BeautifulSoup 获取所有按钮的列表，但未显示所需按钮

Question

I'm using BeautifulSoup to get a list of all button objects on a page, so that I can pick one to later interact with. I'm using the below code, but on the website the add to cart button isn't showing up. If I do inspect element, I can see that it is a button.

I am attempting to do this in as universal a way as possible, so that I can use the same code for multiple websites. So ideally I don't want to use specific attributes to find that button, I want to create a function that allows me to easily find any needed button on any page by displaying all of them and later narrowing it down.

Any idea why I'm not able to find the add to cart button in this code?

from bs4 import BeautifulSoup

page = requests.get('https://www.target.com/p/madden-nfl-21-xbox-one-series-x/-/A-79800769')
data = page.text
soup = BeautifulSoup(data, 'html.parser')
buttons = soup.find_all('button')
for button in buttons:
    print(button)

Answer 1

You will need selenium because buttons are generated by JS. Please read this answer for more explanations.

Code:

from selenium import webdriver
from bs4 import BeautifulSoup
from selenium.webdriver.chrome.options import Options

# start chrome in headless (not visible) mode
options = Options()
options.headless = True
driver = webdriver.Chrome(options=options)

# go to url and wait until page is loaded
url = 'https://www.target.com/p/madden-nfl-21-xbox-one-series-x/-/A-79800769'
driver.get(url)

# get html of page and close webdriver
html = driver.page_source
driver.quit()

# do whatever you want with buttons
soup = BeautifulSoup(html, 'html.parser')
buttons = soup.find_all('button')
print(*buttons, sep='\n')