[英]How can I wait until a specific drop-down menu selected using Selenium Python?
I made a crawler for this page ( http://www.bobaedream.co.kr/cyber/CyberCar.php?gubun=I ) to collect the stock list of specific manufacturers. 我为此页面( http://www.bobaedream.co.kr/cyber/CyberCar.php?gubun=I )制作了一个爬虫,以收集特定制造商的库存清单。 The process is to start from selecting the drop-down menu in the first row of upper part of search menu.
该过程从选择搜索菜单上部第一行的下拉菜单开始。
Each right drop-down menu is child menu of its left drop-down menu. 每个右下拉菜单都是其左下拉菜单的子菜单。 What I would like to do is to select each first item in each drop-down menu and click the "search" button for the first run.
我想做的是在每个下拉菜单中选择每个第一项,然后单击“搜索”按钮进行首次运行。 After crawling of its stock list, then I set the second item of the last drop-down menu and click the "search" button.
爬取其库存清单后,然后设置上一个下拉菜单的第二项,然后单击“搜索”按钮。
But the problem is occurred here. 但是问题发生在这里。 I saved each items of each drop-down menu as tuple.
我将每个下拉菜单的每个项目保存为元组。 When I try to call the second item of the last drop-down menu for the second round of crawling, "StaleElementReferenceException" or "NoSuchElementException" is occurred with the message of "Element is no longer attached to the DOM".
当我尝试为第二轮爬网调用上一个下拉菜单的第二个项目时,发生“ StaleElementReferenceException”或“ NoSuchElementException”,并出现“元素不再连接到DOM”消息。 Thus, I would like to make the element wait until the entire round of each drop-down iteration is completed.
因此,我想让元素等待,直到每个下拉迭代的整个回合完成为止。
Below is my code, but still have the error message. 下面是我的代码,但仍然有错误消息。 My error usually occurs at the second while loop.
我的错误通常发生在第二个while循环中。 At this moment, I guess some type of "wait.until(EC.~)" code in the second "try" function can work this out, but I have no specific idea for this.
目前,我猜想第二个“ try”函数中的某种类型的“ wait.until(EC。〜)”代码可以解决此问题,但是我对此没有具体想法。 Please help or give me any advice.
请帮助或给我任何建议。
def option2_menu_loaded(inDriver):
path = '//select[@id="level2_no"]'
return inDriver.find_element_by_xpath(path)
self.wait.until(option2_menu_loaded)
while True:
try:
select_option2_values = [
('%s' % o.get_attribute('text'), '%s' % o.get_attribute('value'))
for o
in self.getNewSelect("#level2_no").options
if o.get_attribute('text') != '세부등급']
except (StaleElementReferenceException, NoSuchElementException):
print("Exception Found")
continue
break
for option2 in select_option2_values:
self.csv.setCarTitle(ma, mo, de, option1[0], option2[0])
print(option2[0], option2[1])
self.driver.implicitly_wait(0.5)
while True:
try:
self.getNewSelect("#level2_no").select_by_value(option2[1])
except (StaleElementReferenceException, NoSuchElementException):
self.getNewSelect("#level2_no").options
print("Exception Found")
continue
break
If you google the StaleElementException you will see solutions that try to find again the element within a loop. 如果您搜索StaleElementException,将看到解决方案,尝试在循环内再次查找元素。 So that is one idea , in your exception above try 3 times with 1 sec delay before each try to find_Element again, see if this helps.
因此,这是一个主意 ,在上述例外情况下,请尝试3次,并延迟1秒,然后再尝试每次find_Element,这是否有帮助。
Another idea is to refresh the page (certainly not ideal but it might work) between every crawl. 另一个想法是在每个爬网之间刷新页面(肯定不理想,但可能会工作)。 You can do this in Python using:
您可以使用以下命令在Python中执行此操作:
driver.refresh()
Finally, you can also avoid looping (this might be causing the StaleElementException) through all the different elements when crawling as Selenium has a solution for that. 最后,您也可以避免在爬网时通过所有不同的元素循环(这可能导致StaleElementException),因为Selenium有解决方案。 You can save eveything in tuple/array without looping through each record by using find_ElementS instead of find_ElemenT .
您可以使用 find_ElementS而不是find_ElemenT来将eveything保存在元组/数组中,而无需遍历每个记录。 Try this see if it improves your overall performance:
尝试看一下是否可以改善整体效果:
a=[];
a = driver.find_elements_by_xpath(path)
Best of luck! 祝你好运!
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.