How to find the max xpath value in python

Question

I'm working on scraping data in Selenium using xpaths. I want to iterate across xpaths with differing numeric values. For example, I want to iterate across the following xpath: "//*[@id="content-core"]/div/p[i]" with i being the values 1 to n .

Is there a function or code to find n , the maximum number in a given Xpath equation, so I know when to stop the loop?

Thanks so much!

Answer 1

You can use a try-except block, like this:

from selenium.common.exceptions import NoSuchElementException

i = 1

while True:
    try:
        xpath = f"//*[@id="content-core"]/div/p[{i}]"
        driver.find_element_by_xpath(xpath)
        i += 1

    except NoSuchElementException: 
        break

Answer 2

You can use count to solve it. For example, assume if you have extract an html of an url , as follows:

import requests
from lxml import etree

response = requests.get(url)
data = response.text
html = etree.HTML(data)

Then, you can get the length of the label p .

len = int(html.xpath('count(//*[@id="content-core"]/div//p)'))

Answer 3

Find the elements len and then loop through them.

n=len(driver.find_elements_by_xpath("//*[@id='content-core']/div/p"))
for i in range(n):
  try:
    driver.find_element_by_xpath("//*[@id='content-core']/div/p["+i+"]")
  except:
    break