[英]Detected on Headless Chrome Selenium
Im trying to scrape Amazon when running headless with a up to date UserAgent I am getting rate limited.我试图在使用最新的 UserAgent 无头运行时刮掉亚马逊,但我的速率受到限制。 When I # out the headless line I don't get detected or rate limited.
当我 # 退出无头线路时,我不会被检测到或速率受限。 Below is my code!
下面是我的代码!
options = Options()
options.add_experimental_option("excludeSwitches", ["enable-logging"])
options.add_experimental_option("useAutomationExtension", False)
#options.add_argument("--headless")
options.add_argument("--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36")
service = ChromeService(executable_path=ChromeDriverManager().install())
browser = webdriver.Chrome(service=service, options=options)
browser.get("https://www.amazon.com.au/Oculus-Quest-2-Virtual-Reality-Headset/dp/B08FSZ8QWH")
print(browser.page_source)
When I remove the # at options.add_argument("--headless") and I run the code I get a server busy line.当我在 options.add_argument("--headless") 删除 # 并运行代码时,我得到一个服务器忙线。 Does anyone know of a fix?
有谁知道修复?
To avoid detection using google-chrome-headless add the following argument through add_argument()
as follows:为避免使用google-chrome-headless进行检测,请通过
add_argument()
添加以下参数,如下所示:
Sample Code:示例代码:
options = Options()
options.headless = True
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('excludeSwitches', ['enable-logging'])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument('--disable-blink-features=AutomationControlled')
s = Service('C:\\BrowserDrivers\\chromedriver.exe')
driver = webdriver.Chrome(service=s, options=options)
driver.get("https://www.amazon.com.au/Oculus-Quest-2-Virtual-Reality-Headset/dp/B08FSZ8QWH")
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.