如何使列表理解具有“或？”

Question

I am trying to get the list of links from a Google search:

def google_word(word):
    headers={'User-agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/18.17763'}
    url = 'https://google.com/search?q={}'.format(word)
    res= requests.get(url, headers=headers)
    tree= html.fromstring(res.text)
    li = tree.xpath("//a[@href]") #list of links that conatin href
    y = [link.get('href') for link in li if link.get('href').startswith("https://") if "google" not in link.get('href')]

Now, this code collects the right link that starts with " https://" , what I want to do is add the "http://" as well. What do I need to add to the list comprehension in order to make that work (I am trying to do it in one line)?

Answer 1

将元组添加到startswith

y = [link.get('href') for link in li if link.get('href').startswith(("https://", "http://")) if "google" not in link.get('href')]

Answer 2

This line:

y = [link.get('href') for link in li if link.get('href').startswith("https://") if "google" not in link.get('href')]

Should be the below instead:

y = [link.get('href') for link in li if link.get('href').startswith(("https://", "http://"))]

Answer 3

You can use regex to do this. Here's how:

y = [link.get('href') for link in li if re.match("https*://", link.get('href')) if "google" not in link.get('href')]

This will match from zero to unlimited number of occurrences of s (there will be 0 or 1 in real situations).

Answer 4

If you are looking for a way to get search results from google, I would suggest you to use the googlesearch library itself.

It would be much easier for you to get the results. There is no need of scraping the entire query page and search for getting results. It provides you with both http and https links. Here's an article that might be helpul to you.

https://www.geeksforgeeks.org/performing-google-search-using-python-code/