硒加載時間錯誤-尋找可能的解決方法

Question

我正在嘗試從某個網站抓取數據。 我正在使用Selenium，以便可以登錄自己，然后開始解析數據。

我有3個主要錯誤：

• 最后一頁＃無法正確加載。 在這里，我正在加載應為“ 197”的“ 1”，並且我相信這是由於與網站相關的負載而發生的
• 找不到元素'test'xpath。 我在last for循環中注釋掉了。

---

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[1]/div[@class='col-lg-3 col-sm-3 result-info' and 2]/span[@class='brand-name' and 1]"}

---

• 最后，我嘗試單擊最后一頁以測試該方法是否有效，但出現一個錯誤，提示找不到元素。

---

selenium.common.exceptions.ElementNotVisibleException: Message: element not visible

---

這是我的代碼

import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException

url = "https://marketplace.refersion.com/"
username = "jupoxar@b2bx.net"
password = "testpass123"
driver = webdriver.Chrome("/Users/xxx/Downloads/chromedriver")

if __name__ == "__main__":
   driver.get(url)

   driver.find_element_by_xpath("/html/body/div[@class='wrapper']/div[@class='top-block']/header[@class='header clearfix']/div[@class='login-button']/a[@class='login-link']").click()

   driver.find_element_by_id("email").send_keys(username)  # enters the username in textbox

   driver.find_element_by_xpath("/html/body/div[@id='app']/div[@class='top-block']/div[@class='row']/div[@class='col-xs-12 col-sm-10 col-sm-offset-1 col-md-8 col-md-offset-2 col-lg-6 col-lg-offset-3 main-section']/div[@class='main-section-content']/div/form[@class='form-horizontal']/div[@class='form-group ']/div[@class='col-xs-12 col-sm-10 col-sm-offset-1 input-group input-group-lg']/input[@id='password']").send_keys(password)  # enters the password in textbox

   # Find the submit button using class name and click on it.
   driver.find_element_by_class_name("btn-primary").click()

   driver.find_element_by_link_text("Find Offers").click()

   driver.find_element_by_id("sorting-dropdown").click()  # enters the username in textbox

   driver.find_element_by_link_text("Newest First").click()

   last_page = driver.find_element_by_class_name("right-center").text
   print(last_page)

   # try:
   #     last_page = WebDriverWait(driver, 3).until(EC.presence_of_element_located((By.CLASS_NAME, 'right-center')))
   #     print("Page is ready!")
   # except TimeoutException:
   #     print("Loading took too much time!")

   for i in range(1, 10):
     #   test = driver.find_element_by_xpath("//div[1]/div[@class='col-lg-3 col-sm-3 result-info' and 2]/span[@class='brand-name' and 1]")
      #  print(test)

     WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME, 'hover-link'))).click()

---

我認為這與頁面加載方式有關。 我的問題是，是否可以解決類似問題？

Answer 1

您應該在代碼中進行明確的等待 ，以處理頁面的動態加載。 按“最新優先”對頁面進行排序會使頁面刷新結果，並引入微調框以指示排序。

<i class="fa fa-spinner fa-spin" aria-hidden="true" style="font-size: 48px;"></i>

等待微調框消失，應為您提供正確的頁數。 以下幾行內容：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
.....
# your login code
.....

driver.find_element_by_link_text("Newest First").click()
element = WebDriverWait(driver, 10).until(
    EC.invisibility_of_element_located((By.XPATH, "//i[@class='fa fa-spinner fa-spin']"))
)
last_page = driver.find_element_by_class_name("right-center").text

要查找頁面上列出的所有品牌名稱，您需要通過調用方法find_elements_by_xpath （復數，元素）來找到所有具有class='brand-name'的span標簽

brand_names_list = driver.find_elements_by_xpath("//span[@class='brand-name']")
for brand_name in brand_name_list:
    print brand_name.text