Python selenium - get element by its xPath and access it

Question

I am using Python and Selenium to scrape a webpage, in some cases, I can't get it to work, *

I would like to access the element with text 'PInt', wich is the second link in the below code. The xPath for it (copied from Developer console) is: // [@id="submenu1"]/a[2]

      <div id="divTest" onscroll="SetDivPosition();" style="height: 861px;">
            <div class="menuTitle" id="title1">
                <a href="#" onclick="toggle(1);"> </a>
            </div>
            <div class="subtitle" id="submenu1">    
                <img src="images/spacer.gif" border="0" width="2px" height="12px">
                <a href="#" class="NormalBlueSmall" onclick="clickItem('area/search/mov/mov2','mov');">Mov</a><br>
                <img src="images/spacer.gif" border="0" width="2px" height="12px">
                <a href="#" class="NormalBlueSmall" onclick="clickItem('area/con/ExtInt/extInt','pIint');">PInt</a><br>
                <img src="images/spacer.gif" border="0" width="2px" height="12px">
                <a href="#" class="NormalBlueSmall" onclick="clickItem('GoToNew.asp?link=asw_cnt/SmanSwif.aspx','SMAN/SWIF');">SWAM / SWIF</a><br>
            </div>


...

A snipped of my code is:

try:
    res = driver.find_elements_by_link_text('PInt')
    print("res1:{}".format(res))

    res = driver.find_element(By.XPATH,'//*[@id="submenu1"]/a[3]')
    print("res:{} type[0]:{}".format(res,res[0]))
    itm1 = res[0]
    itm1.click()

I get the error:

Unable to locate element: {"method":"xpath","selector":"//*[@id="submenu1"]/a[2]"}

My question is, how can I get the right xPath of the element or any other way to access the element?

UPDATE: This might be important, the issue with Message: invalid selector: Unable to locate an element with the xpath expression (and I've tried all proposed solutions) might be that this is after authenticate in the webpage (User + Pwd) before, everything works. I noticed that the url driver.current_url after login is static ( asp page). Also this part I am trying to access in a frameset and frame

html > frameset > frameset > frame:nth-child(1)

Answer 1

Thanks to @JeffC to point me in the right direction.

since the page has some frames, I manage to access the element first by switching to the right frame (using xPath) and then access the element.

driver.switch_to.default_content()
driver.switch_to.frame(driver.find_element_by_xpath('html / frameset / frameset / frame[1]'))

driver.find_element_by_xpath("//a[contains(text(),'PInt')]").click()

BTW, in case you want to run the script from a crontab ,you need to setup a display:

30 5 * * * export DISPLAY=:0; python /usr/.../main.py

Answer 2

To see a full list of all the ways of selecting elements using selenium, you can read all about it in the documentation .

Using xpath:

res = driver.find_element_by_xpath(u'//*[@id="submenu1"]/a[2]')

Using css selector:

res = driver.find_element_by_css_selector('#submenu1 a:nth-of-type(2)')

Answer 3

Try with any of the below xpath. Sometimes automatically generated xpath does not work.

//a[contains(text(),'PInt')]
or
//div[@id='submenu1']//a[contains(text(),'PInt')]

Also I would suggest you to set some wait time before clicking on above link in case if above xpath does not work

Answer 4

To find xPath in chrome:

1. Right click the element you want
2. Inspect, which will open the developer window and highlight the selected element.
3. Right click the highlighted element and choose copy > copy by xpath

Here is a list of all the different ways to locate an element Locating Elements