Interacting with JavaScript object when web scraping with python and selenium

Question

I am trying to load more comments, which is loaded by clicking on a JavaScript object. Then scrap the page. To test this out, i'm printing the number of comments (inside ap tag) the page contains before and after clicking on the "load more" button. But, it points the same number of p tags before and after. If if you look at the page it has a lot more comments. Where am I going wrong?

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

driver = webdriver.PhantomJS(executable_path='PATH_TO.../phantomjs')
driver.get('http://www.ratemyprofessors.com/ShowRatings.jsp?tid=1500075')

comments = driver.find_elements_by_tag_name('p')
print('Before', len(comments))

time.sleep(1)


try:
    element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, 'loadMore')))
    time.sleep(1)

finally:
    comments = driver.find_elements_by_tag_name('p')
    print('After', len(comments))

driver.close()

Note that I tried both 'loadMore' and 'loadmoreBlog'.. did not work. Thanks so much in advance for your answers.

Answer 1

我需要添加

element.click()