Unable to click button - Python / Selenium & bs4

Question

Im trying to click a button and tried already several different methods with no avail.

Method 1:

WebDriverWait(driver, 20).until(EC.invisibility_of_element((By.CLASS_NAME, 'sc-gsDKAQ fWOgSr')))
driver.execute_script("arguments[0].click();", WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CLASS_NAME, 'sc-gsDKAQ fWOgSr'))))

Method 2:

driver.find_element(by=By.CLASS_NAME, value='sc-gsDKAQ fWOgSr').click()

Method 3:

btn = driver.find_element(by=By.CLASS_NAME, value='sc-gsDKAQ fWOgSr')
btn.click()

Error messages:

M1: Timeout

M2: NoSuchElementException

M3: NoSuchElementException

I tried the same by looking with Xpath, no success. Had another Error in the past, which stated that "String" cannot be interacted with.

Answer 1

That element is in a #shadow root element, so it's a bit fiddly to reach. Also, page seems to react at mouse movements (and only load after it detect some mouse movement). The following code seems to work: setup is on Linux, but you can adapt it to your own, just observe the imports and the part after defining the browser:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains


chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument('disable-notifications')
import time as t


webdriver_service = Service("chromedriver/chromedriver") ## path to where you saved chromedriver binary
browser = webdriver.Chrome(service=webdriver_service, options=chrome_options)

actions = ActionChains(browser)

url = 'https://www.immobilienscout24.at/regional/wien/wien/wohnung-kaufen'

browser.get(url) 

page_title = WebDriverWait(browser, 3).until(EC.presence_of_element_located((By.CSS_SELECTOR, "a[title='Zur Homepage']")))
actions.move_to_element(page_title).perform()
parent_div = WebDriverWait(browser, 20000).until(EC.presence_of_element_located((By.ID, "usercentrics-root"))) 
shadowRoot = browser.execute_script("return arguments[0].shadowRoot", parent_div)

try:
    button = WebDriverWait(shadowRoot, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[data-testid='uc-accept-all-button']"))) 
    button.click()
    print('clicked')
except Exception as e:
    print(e)
    print('no click button')

This will click the button, and print in the terminal:

clicked