使用python在网页上单击“加载更多”

Question

I am trying to load the whole content of this link (clicking on المزيد)

I tried so many tutorials like this one here , which teaches how to work with a similar issue (Infinite Scrolling Pages).

My issue is that I couldn't manage to specify the load more class on this page to click it. But, if I am not mistaken, it exists in this part of the webpage source code:

<ng-include src="'loader.html'" class="ng-scope">
    <div class="loading-div ng-scope ng-hide" ng-show="!loadingMoreData">
        <div class="spinner">
            <div class="bounce1"></div>
            <div class="bounce2"></div>
            <div class="bounce3"></div>
        </div>
    </div>
</ng-include>
<div class="block-hed block-link clearfix" data-ng-show="isMoreDisplayed" data-ng-click="getMoreMaterials()">
                    <h4 class="link-border-color"><a href="javascript:void(0)">المزيد </a></h4>
                </div>

I do not necessarily need to implement any function like "click()" or "perform()". Any way to show the whole content under load more button is considered.

---

by the way, this is my code so far:

from selenium import webdriver
browser = webdriver.Chrome("/home/aziz/anaconda3/lib/python3.6/site-packages/chromedriver/chromedriver")
am = browser.get("https://sabq.org/%D8%A7%D9%84%D9%85%D9%85%D9%84%D9%83%D8%A9/%D8%A5%D9%82%D8%AA%D8%B5%D8%A7%D8%AF")

---

UPDATE I used this link here to solve the issue, but none of the tutorials worked.

find_element_by_id
find_element_by_name
find_element_by_xpath
find_element_by_link_text
find_element_by_partial_link_text
find_element_by_tag_name
find_element_by_class_name
find_element_by_css_selector

Answer 1

I finally managed to solve the problem. I just added this line after my code

while True:
    browser.find_elements_by_link_text('المزيد')[1].click()

and the page started infinitely loading all of the articles. I didn't really know that المزيد itself is clickable. I thought there is a link included in it.

Answer 2

The correct/better way to do it is to use WebDriverWait

from selenium.webdriver.support import expected_conditions as EC

wait = WebDriverWait(driver, 10)
while True:
  element = wait.until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, 'المزيد')))
  element.click()