Selenium + bs4 在抓取之前选择正确的数据

Question

I want to scrape a data from this site https://www.marketbeat.com/insider-trades/ceo-share-buys-and-sales/ . Problem is i want to choose some options before scraping

In the Country button i want to check every button

This is default state

After scraping data i want to repeat this process but with

Transaction type Sold shares checked

For now i have https://colab.research.google.com/drive/1VUVSI5BGiN5lCFrj4WGfJaZg1-wtU2f1?usp=sharing

If someone could give me some hints i would apprecate that thanks

Answer 1

This should work. You have to click the labels rather than the checkboxes

dropdownButton = driver.find_element(By.ID, 'dropdownCountry');
dropdownButton.click();

dropdownList = driver.find_element(By.XPATH, '//ul[@aria-labelledby="dropdownCountry"]')
countryButtons = dropdownList.find_elements(By.TAG_NAME, "label");
countryButtons.pop(0)
for i in countryButtons:
    i.click()

soldSharesButton = driver.find_element(By.XPATH, '//option[@value="Sell"]')
soldSharesButton.click()