Python + Selenium — Cannot access elements on a webpage

Question

I am trying to access an element in Selenium, but none of the typical methods are working. So far I have tried using every variation of 'find_elements_by_x' with no success. I also spent about a day looking at various forums, but nothing seems to be working. I recently used Selenium in another successful project, but the same structure is not working for this particular website. Here is a snippet of the HTML containing the elements I am trying to access:

<input type="text" name="username" id="username" placeholder="Username / 
Email" autocapitalize="off" autocorrect="off" required="" ng-
model="credentials.username" class="ng-pristine ng-invalid ng-invalid-
required">

As is probably obvious, this is a username input for logging in. Below are a few lines I have tried so far that have not worked.

from selenium import webdriver


driver = webdriver.Chrome("chromedriver.exe filepath")
driver.get('url')

username = driver.find_element_by_xpath('//input[@id="username"]')

That XPath navigates to the element in question if I search for it using the console in Chrome.

I also tried:

username = driver.find_element_by_name('username')

which also did not work.

I am pretty new to Selenium, and I have no experience at all with HTML, so I don't know if there can be complications in the HTML that that must be taken into account when looking for elements through Selenium. Any help at all is appreciated. This is also my first time posting here, so I hope I did not violate any rules.

Answer 1

Looks like the input box is taking a few seconds to load. Try putting a delay before your request:

driver.implicitly_wait(3)

If that doesn't work, increase the number in parentheses a second at a time. If that solves the problem, you could instead do an explicit wait.

Answer 2

Try the following code:

from selenium import webdriver
from selenium.webdriver.support import ui
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By


options = Options()
options.add_argument("unlimited-storage")
driver = webdriver.Chrome(chrome_options=options)

driver.get("http://www.runescape.com/companion/comapp.ws")

iframe = ui.WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.TAG_NAME, "iframe")))

# Switch to the frame.
driver.switch_to.frame(iframe)

username = ui.WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.ID, "username")))

print(username)

# Switch to the main content.
driver.switch_to.default_content()

driver.quit()

Hope it helps you!

Answer 3

As per the HTML you have shared the WebElement is a Angular element so you have to induce WebDriverWait for the element as follows :

• CSS_SELECTOR :

   username = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.ng-pristine.ng-invalid.ng-invalid-required#username"))) 

• XPATH :

   username = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@class='ng-pristine ng-invalid ng-invalid-required' and @id='username']")))