为什么我的 for 循环在我的列表中途失败？ Selenium webdriver，driver.find_element_by_xpath

Question

I have a list of 10 linkedin urls that I am running. For 8 of the urls, the code works perfectly. But it consistently fails during the 9th iteration.

This is my traceback:

Traceback (most recent call last):
  File "/Users/yu/PycharmProjects/linkedin/hello.py", line 97, in <module>
    images = driver.find_element_by_xpath('// img[ @ id = "ember51"]')
  File "/Users/yu/PycharmProjects/linkedin/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath
    return self.find_element(by=By.XPATH, value=xpath)
  File "/Users/yu/PycharmProjects/linkedin/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element
    return self.execute(Command.FIND_ELEMENT, {
  File "/Users/yu/PycharmProjects/linkedin/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/Users/yu/PycharmProjects/linkedin/venv/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"// img[ @ id = "ember51"]"}
  (Session info: chrome=80.0.3987.122)

Relevant code:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from parsel import Selector

ChromeOptions = webdriver.ChromeOptions()
ChromeOptions.add_argument('--disable-browser-side-navigation')
driver = webdriver.Chrome('/Users/yu/Downloads/chromedriver')

linkedin_urls=driver.find_elements_by_xpath('//*[@id="rso"]/div[1]/div/div')
linkedin_urls=[url.find_element_by_xpath('div/div/div[1]/a').get_attribute('href') for url in linkedin_urls]

for linkedin_url in (linkedin_urls):
    driver.get(linkedin_url)
    sel = Selector(text=driver.page_source)

images = driver.find_element_by_xpath('// img[ @ id = "ember51"]')
print(images.get_attribute('src'))

Why would the code stop working? I checked the 9th url , and the xpath is the same for images

Answer 1

It's failing on

images = driver.find_element_by_xpath('// img[ @ id = "ember51"]')

The element with the id of "ember51" is a div, so change it to this:

images = driver.find_element_by_xpath('// div[ @ id = "ember51"]/img')

This way when you try to get the src attribute on the next line, it will work.