告诉硒在预期条件后停止阻断

Question

I am interested in one element, let's call it

<div class="ofInterest" some-attr="dataIReallyWant"></div>

When I switch off js in firefox, this element does not exist. With javascript it does. I could not tell how it was being generated but my guess is that there is an ajax call which returns a js file which executes this javascript.

I am using selenium but it is very slow. I want to tell Selenium this:

1. Wait for this element to load, ie something like EC.visibility_of_element_located((By.CSS, '.ofInterest'))

2. once you detect said element, stop blocking the code and don't download any further so don't waste my bandwidth

Answer 1

Simply wait for the element to exist in the DOM, then either quit/close the browser or execute some JavaScript to stop the page from loading:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Firefox()
driver.get("http://somedomain/url_that_delays_loading")
try:
    element = WebDriverWait(driver, 10).until( #10 second timeout.
        EC.presence_of_element_located((By.ID, "myDynamicElement"))
    )
finally:
    driver.quit()
    # OR
    #driver.execute_script("window.stop();")

More information can be found here.

Answer 2

Explicit waits were made exactly for what you are describing:

An explicit waits is code you define to wait for a certain condition to occur before proceeding further in the code. The worst case of this is time.sleep(), which sets the condition to an exact time period to wait.

In the worst case scenario, you would wait X amount of seconds that you've passed to the WebDriverWait , 10 seconds in this case:

element = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.ID, "myDynamicElement"))
    )

But, if the element is found earlier, it would give you the element and stop blocking the execution. By default, it checks for the expected condition every 500ms.

FYI, under-the-hood, it is just a while True: loop :

def until(self, method, message=''):
    """Calls the method provided with the driver as an argument until the \
    return value is not False."""
    screen = None
    stacktrace = None

    end_time = time.time() + self._timeout
    while True:
        try:
            value = method(self._driver)
            if value:
                return value
        except self._ignored_exceptions as exc:
            screen = getattr(exc, 'screen', None)
            stacktrace = getattr(exc, 'stacktrace', None)
        time.sleep(self._poll)
        if time.time() > end_time:
            break
    raise TimeoutException(message, screen, stacktrace)