如何使 Selenium python 代码更高效

Question

I have a problem while doing some webscraping, it's I have 3 loops where I go through many links, but the problem is when it's get to the third link the code crashes. or How can I put some waits in my program without using the sleep function.

Answer 1

Seems like what you want to do is wait for element to load. For this selenium provides WebDriverWait() function.

from selenium import webdriver
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()
driver.get('url')
timeout = 5
try:
    element_present = EC.presence_of_element_located((By.ID, 'element_id'))
    WebDriverWait(driver, timeout).until(element_to_be_present)
except TimeoutException:
    print("Timed out waiting for page to load")