I am trying to iterate clicking through the North American cities in the Uber Movement data.
I have setup Python code to select the table of all cities (cities_table), make it into a list, and identify the button for each city to be clicked (the button has a tag name 'a').
My problem is that I am able to select the first city in the table (Atlanta), but when I try to go back to the page with the table of all cities I do not manage to click on the next city on the list, which should be Boston.
After I successfully enter the Atlanta page, I setup a driver.back() to go back to the cities table, but when trying to move on to the next item on the list, I get the following error message:
Message: stale element reference: element is not attached to the page document
How can I correctly click on a city, go back to the cities table page and click on the NEXT item?
Here is my code so far:
# Load Chrome driver and movement.uber.com/cities website
PATH = 'C:\Program Files (x86)\chromedriver.exe'
driver = webdriver.Chrome(PATH)
driver.get('https://movement.uber.com/cities?lang=en-US')
# City loop
cities_table = driver.find_element_by_xpath('/html/body/div[1]/div[1]/div[1]/div[2]/div[1]/div')
cities = cities_table.find_elements_by_tag_name('div')
for city in cities:
city = city.find_element_by_tag_name('a') # Buttons have tags "a"
city.click()
time.sleep(3)
driver.back()
Full error message below:
StaleElementReferenceException Traceback (most recent call last)
<ipython-input-25-a148ef184a6b> in <module>
6 cities = cities_table.find_elements_by_tag_name('div')
7 for city in cities:
----> 8 city = city.find_element_by_tag_name('a') # Buttons have tags "a"
9 city.click()
10 time.sleep(3)
~\AppData\Roaming\Python\Python37\site-packages\selenium\webdriver\remote\webelement.py in find_element_by_tag_name(self, name)
303 element = element.find_element_by_tag_name('h1')
304 """
--> 305 return self.find_element(by=By.TAG_NAME, value=name)
306
307 def find_elements_by_tag_name(self, name):
~\AppData\Roaming\Python\Python37\site-packages\selenium\webdriver\remote\webelement.py in find_element(self, by, value)
657
658 return self._execute(Command.FIND_CHILD_ELEMENT,
--> 659 {"using": by, "value": value})['value']
660
661 def find_elements(self, by=By.ID, value=None):
~\AppData\Roaming\Python\Python37\site-packages\selenium\webdriver\remote\webelement.py in _execute(self, command, params)
631 params = {}
632 params['id'] = self._id
--> 633 return self._parent.execute(command, params)
634
635 def find_element(self, by=By.ID, value=None):
~\AppData\Roaming\Python\Python37\site-packages\selenium\webdriver\remote\webdriver.py in execute(self, driver_command, params)
319 response = self.command_executor.execute(driver_command, params)
320 if response:
--> 321 self.error_handler.check_response(response)
322 response['value'] = self._unwrap_value(
323 response.get('value', None))
~\AppData\Roaming\Python\Python37\site-packages\selenium\webdriver\remote\errorhandler.py in check_response(self, response)
240 alert_text = value['alert'].get('text')
241 raise exception_class(message, screen, stacktrace, alert_text)
--> 242 raise exception_class(message, screen, stacktrace)
243
244 def _value_or_default(self, obj, key, default):
StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
(Session info: chrome=84.0.4147.89)
This error occurs because you are getting the reference of a dom element which no longer exists. Every time a page is reloaded or refreshed, or the dom has somewhat changed, every reference you have previously obtained is useless.
So, in order to work, you have to keep a reference of what you have already processed, and renew your dom reference every time the page changes.
# City loop
managed_cities = []
while True:
cities_table = driver.find_element_by_xpath('/html/body/div[1]/div[1]/div[1]/div[2]/div[1]/div')
cities = cities_table.find_elements_by_tag_name('div')
for city in cities:
# Here you have to find and ID or some unique text that identifies the
# already processed city in order to skip it
if city.text in managed_cities:
continue
city = city.find_element_by_tag_name('a') # Buttons have tags "a"
city.click()
time.sleep(3)
driver.back()
break
# Now, if you have managed all the cities, you can exit from the while
# otherwise, keep loop until all cities have been processed
if len(managed_cities) == len(cities):
break
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.