Selenium python, Click only under certain conditions

Question

My code works, but not in all cases Basically the functionality is to click a load_more button until it no longer appears.

As of right now, I simply have a loop which finds the loadmore button and clicks it twice, but there are cases that it will click on something else when the load more button disappears.

I was planning on making a while loop, which would constantly find the click the load_more option until the loadmore disappeared then break the loop.

Here is the code: (This simply finds and clicks it twice)

load_more = browser.find_element_by_css_selector("#mainContent > div.left-panel > div > div.result-list > div > div.content")
WebDriverWait(browser, timeout).until(EC.visibility_of(load_more))

#Need bugfix, 
for i in range(2):
    browser.execute_script("return arguments[0].scrollIntoView(true);", load_more)
    ActionChains(browser).move_to_element(load_more).click().perform()

I noticed when playing around with the load more button that.

<div class="progressbtnwrap" data-search-type="search" style="display: block;">

When the load more button is present on the site, the element is set to "display: block;"

But once the load more button disappears,

<div class="progressbtnwrap" data-search-type="search" style="display: none;">

the element changes to none, notice "display: none;"

Any suggestions how I can search for this?

When looking through the selenium documentations I couldn't find any way of searching for this element and specifically checking if the style is triggered to none,

https://selenium-python.readthedocs.io/locating-elements.html

My goal here is to create something like this

while(True):
    if browser.find_element_by_notsurewhat == "block":
        ActionChains(browser).move_to_element(load_more).click().perform()
    if browser.find_element_by_notsurewhat == "none":
        break
    browser.execute_script("return arguments[0].scrollIntoView(true);", load_more)

I'm sure the logic must be much more complicated than that, or even if what I want to achieve is even possible, Any suggestions would be amazing!

Thank you all!

UPDATE:

def load_more(browser):
    print("I'm in the function LOAD MORE")
    try:
        if browser.find_element_by_xpath('//*[@id="mainContent"]/div[1]/div/div[5]/div'):
            print("I HAVE ENTERED THE TRY BLOCK WITHIN THE LOAD MORE FUNCTION")
            return True
    except Exception as e:
        print(e)
        return False
    return False
while load_more(browser):
    print("I'm in the while loop!")
    ActionChains(browser).move_to_element(load_more).click().perform()
    browser.execute_script("return arguments[0].scrollIntoView(true);", load_more)

When placing my locating and clicking commands, I began to receive the following error:

Traceback (most recent call last):
  File "C:\Users\David\eclipse-workspace\Web_Scrap\setup.py", line 81, in <module>
    ActionChains(browser).move_to_element(load_more).click().perform()
  File "C:\Users\David\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\common\action_chains.py", line 83, in perform
    action()
  File "C:\Users\David\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\common\action_chains.py", line 293, in <lambda>
    Command.MOVE_TO, {'element': to_element.id}))
AttributeError: 'function' object has no attribute 'id'

I noticed from trying to figure out exactly where the program crashes that, once the code below is run, the program crashes, but this works prior to placing this inside the while loop, or the function. (I tried to place the scrollIntoView, line inside the function right before the try, and I receive a similar error).

ActionChains(browser).move_to_element(load_more).click().perform()
browser.execute_script("return arguments[0].scrollIntoView(true);", load_more)

Answer 1

The idiomatic way to do this is to use "explicit waits" (AKA WebDriverWait with ExpectedConditions ).

The following will wait until the element is no longer visible. If it doesn't disappear in 10 secs, a TimeOutError is raised:

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

wait = WebDriverWait(driver, 10)
element = wait.until(EC.invisibility_of_element_located(By.CLASS_NAME, 'progressbtnwrap'))

Answer 2

If the style attribute for an html element is set to display: none, selenium won't be able to find the element using the built-in DOM selector functions like find_element_by_id/find_elements_by_class etc. You could simply wrap the find operation in a try except block and add a delay to allow the browser some time for the Ajax call.

def load_more(browser):
    time.sleep(1)
    try:
        display = browser.execute_script("return document.getElementsByClassName('progressbtnwrap')[0].style.display")
        if display == 'none':
            return False
        elem = browser.find_element_by_xpath('//div[contains(@class, "progressbtnwrap")]/div[contains(@class, "content")]')
        browser.execute_script("arguments[0].click();", elem)
        return True
    except Exception as e:
        print("Error")
        print(e)
    return False

while load_more(browser):
    print("scrolling further")

Answer 3

Assuming you are currently just trying to find a way on how you can check the current style of your element you can use this code.

driver.execute_script("return arguments[0].style.display;", load_more)

And you can use to check that when the return value is 'none' for a few seconds which means no more data will be loaded, you can exit your loop.