使用 Python 中的 Selenium 从“结果页面”中提取结果

Question

Unfortunately I have a little problem with the realization of my Python program. At one point I can't get any further. The program should do the following:

1. Perform an automatic search for a specific keyword on the search engine "www.startpage.com".
2. Then the page with the results should be read out (and here is the problem).
3. The program should now count how often a certain word appears on the page with the search results.

The problem here is that I can't get the source code from the search results page. I only get the source code of the start page Does anyone know a solution?

Thanks in advance.

So far my program looks like this:

import selenium.webdriver as webdriver

def get_results(search_term):

    #this is the site, where I want to do the search
    url="https://www.startpage.com"
    browser = webdriver.Firefox()
    browser.get(url)

    search_box = browser.find_element_by_id("q")
    #search in the search box after the search term
    search_box.send_keys(search_term)
    search_box.submit()

    #print(browser.page_source) would give the result of the startpage (not the result page)

    sub="dog"
    print(source_code.count("dog"))
    #counts zero times because it searchs for "dog" at the startpage

get_results("dog")

Answer 1

You can do it this way: Just make a loop in which you always add an elment to a list (can be a number or a letter, for example) when this term is found.

To do this you have to save the source code in a variable and then simply search for the term in it. When it is found you simply add a number to a list with .append() and then at the end you check the length of the list with len(list) .