Web crawling - how to select path of element when selecting it doesn't change anything in the URL or xpath?

Question

I am trying to select "Custom range" in the date selector of this site: https://niftygateway.com/sitewide-activity

I have written the following code to make Selenium click on "Right now":

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
import time

webdriver = webdriver.Chrome(ChromeDriverManager().install()) #executable_path=chromedriver_path
time.sleep(2)
webdriver.get('https://niftygateway.com/sitewide-activity')
time.sleep(3)

date_selector = webdriver.find_element_by_xpath('//*[@id="root"]/div/div/div[1]/div[2]/div/div/button/span')
date_selector.click()

But after this, how do I make it click on "Custom range"? When I "inspect element" I cannot see anything change in the path upon selecting "Custom range". So I am not able to find a way to automate a click on "Custom range".

Furthermore, how do I make it select a date range, say Nov 1 or Nov 5? Because when I select a custom date range myself, again nothing in the xpath or URL changes, so I am not sure how to automate this.

Answer 1

You can target input fields that has value attribute to send the dates.

Code:

driver = webdriver.Chrome(driver_path)
driver.maximize_window()
driver.implicitly_wait(30)
wait = WebDriverWait(driver, 30)

driver.get("https://niftygateway.com/sitewide-activity")

wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[aria-controls='simple-menu']"))).click()
time.sleep(2)

wait.until(EC.element_to_be_clickable((By.XPATH, "//p[text()='Custom range']//following-sibling::div"))).click()

wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.MuiDialogContent-root")))

time.sleep(2)
start_to = wait.until(EC.element_to_be_clickable((By.XPATH, "//label[text()='Start Date']//following-sibling::div/input")))
driver.execute_script("arguments[0].setAttribute('value', '11/03/2021')", start_to)

time.sleep(2)
end_date = wait.until(EC.element_to_be_clickable((By.XPATH, "//label[text()='End Date']//following-sibling::div/input")))
driver.execute_script("arguments[0].setAttribute('value', '11/12/2021')", end_date)

time.sleep(2)
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[text()='Save']"))).click()

Answer 2

To click on Custom range you need to induce WebDriverWait for the element_to_be_clickable() and you can the following Locator Strategies :

driver.get("https://niftygateway.com/sitewide-activity")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@aria-controls='simple-menu']/span"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//p[text()='Custom range']"))).click()