Python打印结果中包含特定的字符串

Question

I am trying to get google search result description.

from selenium import webdriver
import re
chrome_path = r"C:\Users\xxxx\Downloads\Compressed\chromedriver_win32\chromedriver.exe"
driver = webdriver.Chrome(chrome_path)
driver.get("https://www.google.co.in/search?q=stackoverflow")
posts = driver.find_elements_by_class_name("st")
for post in posts:
    print(post.text)

Here Im getting correct results. But I only want to print links from description. And want to get results from 5 google search pages. Here I am only getting from 1 page.

I have tried using

print(post.get_attribute('href'))

but description links are not clickable so this returns None.

Answer 1

Try the below code:

for i in range(1, 6, 1):
    print("--------------------------------------------------------------------")
    print("Page "+str(i)+" Results : ")
    print("--------------------------------------------------------------------")
    staticLinks = driver.find_elements_by_xpath("//*[@class='st']")
    for desc in staticLinks:
        txt = desc.text+''
        if txt.count('http://') > 0 or txt.count('https://') > 0:
            for c in txt.split():
                if c.startswith('http') or c.startswith('https'):
                    print(c)

    dynamicLinks = driver.find_elements_by_xpath("//*[@class='st']//a")
    for desc in dynamicLinks:
        link = desc.get_attribute('href')
        if link is not None:
            print(link)
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    nextPage = driver.find_element_by_xpath("//a[@aria-label='Page "+str(i+1)+"']");
    nextPage.click();

Will try to fetch the static & dynamic links from the google's first 5 search results description.