简体   繁体   中英

How do I request search results from google without trigerring a 429 error( too many requests for url)?

My script makes only one request from 'www.google.com/search?q=' despite this it always triggers a 429 error( too many requests for url). The script simply attempts to use python's 3rd party requests library to retrieve the search page for a word entered as a command line variable. Solutions I have tried include adding a header specifying my user agent.This is all of the code it is quite small.

import requests, sys, webbrowser, bs4

headers = {'User-agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:85.0) Gecko/20100101 Firefox/85.0' }
print('Googling...') # display text while downloading the Google page
res = requests.get('http://www.google.com/search?q=' + ' '.join(sys.argv[1:]), headers=headers)
res.raise_for_status()

The error reads as follows

  File "lucky.py", line 11, in <module>
    res.raise_for_status()
  File "C:\ProgramData\Anaconda3\lib\site-packages\requests\models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url: https://www.google.com/sorry/index?continue=https://www.google.com/search%3Fq%3Dred&q=EgRmsF6JGJ_0-48GIhDND5xins41tXhPPRNDx_-jMgFy

Thanks

Edited***

So the code runs when I use an online compiler but not on my PC...

The code works for me Here<\/a> .

its a security measurement by the server

I figured out a while back that the problem was caused by my OS settings and the solution is a rather simple thing. I'm run windows 10 by the way. All you need to do is Go to settings -> Click on.network & inte.net -> Click VPN (its on the sidebar) -> and turn off the system VPN It worked fine after this. So the problem was the windows default VPN, it doesn't pair well with automated google searches for some reason. Does anyone know why?

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM