简体   繁体   中英

Selenium takes too much time loading a page

I'm trying to crawl whoscored which has a lot of stuff and takes a lot of time to load completely. However, the stuff that I want are loaded pretty fast, but nonetheless driver.get(url) executes until the webpage is loaded completely. Is there any way to prevent this from happenning ? To make the get method return as soon as some element I define is present in the DOM ? I'm thinking something as stop works in a browser.

So, the best solution I found for the kind of behaviour I wanted was:

profile = webdriver.FirefoxProfile()
profile.set_preference("webdriver.load.strategy", "unstable")
profile.update_preferences()
driver = webdriver.Firefox(firefox_profile=profile)
driver.get(url)
try:
    wait = WebDriverWait(driver, timeout=20, poll_frequency=0.1)
    wait.until(<expectation object>)
finally:
    driver.execute_script("return window.stop")

This will stop the browser loading the page and you can still crawl and interact with the loaded site.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM