[英]How can I hold the driver of a specific page until a specific drop-down menu list save and entire list request using Selenium Python?
I made a crawler for this page ( http://www.bobaedream.co.kr/cyber/CyberCar.php?gubun=I ) to collect the stock list of specific manufacturers. 我为此页面制作了一个爬虫( http://www.bobaedream.co.kr/cyber/CyberCar.php?gubun=I )来收集特定制造商的库存清单。 My code starts from selecting the drop-down menu in the first row of search menu.
我的代码从选择搜索菜单第一行中的下拉菜单开始。
Each right drop-down menu is child menu of its left drop-down menu. 每个右下拉菜单都是其左下拉菜单的子菜单。 What I would like to do is to select each first item in each drop-down menu and click the "search" button for the first run.
我想做的是在每个下拉菜单中选择每个第一项,然后单击“搜索”按钮进行首次运行。 After crawling of its stock list, then I set the second item of the last drop-down menu and click the "search" button.
爬取其库存清单后,然后设置上一个下拉菜单的第二项,然后单击“搜索”按钮。 (If there is only one item in the last drop-down menu, then go-up to the second drop-down menu from the last and pick the second item in it.)
(如果最后一个下拉菜单中只有一个项目,则从上一个下拉菜单转到第二个下拉菜单,然后选择其中的第二个项目。)
But the problem is occurred here. 但是问题发生在这里。 I saved each items of each drop-down menu as tuple.
我将每个下拉菜单的每个项目保存为元组。 When I try to call the second item of the last drop-down menu for the second round of crawling, "StaleElementReferenceException" or "NoSuchElementException" is occurred with the message of "Element is no longer attached to the DOM".
当我尝试为第二轮爬网调用上一个下拉菜单的第二个项目时,发生“ StaleElementReferenceException”或“ NoSuchElementException”,并出现“元素不再连接到DOM”消息。 Thus, I would like to make the drive until each drop-down menu iteration is completed.
因此,我想驱动器直到每个下拉菜单迭代完成为止。
Below is my code, but still have the error message. 下面是我的代码,但仍然有错误消息。 My error usually occurs at the second while loop.
我的错误通常发生在第二个while循环中。 At this moment, I added some "wait" function and "refresh" function.
这时,我添加了一些“等待”功能和“刷新”功能。 Sometimes it can help to get out of while loop in my code, but usually it got stuck in the while loop and infinite repeat.
有时它可以帮助我退出代码中的while循环,但是通常它会陷入while循环和无限重复中。 I guess some type of "wait.until(EC.~)" code in the second "try" function can work this out, but I have no specific idea for this.
我猜第二个“ try”函数中的某种类型的“ wait.until(EC。〜)”代码可以解决此问题,但是我对此没有具体想法。 Please help or give me any advice.
请帮助或给我任何建议。
def option2_menu_loaded(inDriver):
path = '//select[@id="level2_no"]'
return inDriver.find_element_by_xpath(path)
self.wait.until(option2_menu_loaded)
while True:
try:
select_option2_values = [
('%s' % o.get_attribute('text'), '%s' % o.get_attribute('value'))
for o
in Select(self.driver.find_element_by_css_selector("#level2_no")).options
if o.get_attribute('text') != '세부등급']
except (StaleElementReferenceException, NoSuchElementException):
print("=======Exception Found - Option2 Save=====")
self.driver.refresh()
self.driver.implicitly_wait(1.5)
continue
break
for option2 in select_option2_values:
self.csv.setCarTitle(ma, mo, de, option1[0], option2[0])
print(option2[0], option2[1])
self.driver.implicitly_wait(0.5)
while True:
try:
Select(self.driver.find_element_by_css_selector("#level2_no")).select_by_value(option2[1])
except (StaleElementReferenceException, NoSuchElementException):
print("=======Exception Found - Option2 Request=====")
self.driver.refresh()
self.driver.implicitly_wait(1.5)
self.driver.refresh()
continue
break
This means the bags reloads between finding the element and using the element object, make sure the page is loaded first and search for the element each time you need to use it. 这意味着在查找元素和使用元素对象之间重新加载包,确保首先加载页面并在每次需要使用元素时搜索该元素。
An extra check would be to add an extra wait for the element before doing the select since find_element_by_css_selector
will not wait at all for element to be available. 额外的检查是在执行选择之前为元素添加额外的等待,因为
find_element_by_css_selector
根本不会等待元素可用。
The sequence should be: page is loaded > wait until element exists > use action on element 顺序应为:页面加载>等待元素存在>对元素使用操作
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.