[英]Access Denied Page with selenium
Im trying to code up a small application that uses footlocker and scrapes certain pages from the website.我正在尝试编写一个使用 footlocker 并从网站上抓取某些页面的小应用程序。 Now the way I want to do the app is by starting at the footlocker homepage and then clicking through different parts on the website.
现在我想要做这个应用程序的方式是从footlocker主页开始,然后点击网站上的不同部分。 Below I have given an example of one of the additional links that I would click to then scrape.
下面我给出了一个附加链接的例子,我会点击然后抓取。 The issue though that I am having is that when the application finds the button and clicks on it I go to a error page kind of and then if I refresh the page I get an Access denied page.
我遇到的问题是,当应用程序找到按钮并单击它时,我会转到一个错误页面,然后如果我刷新页面,我会得到一个拒绝访问的页面。 If anyone could help me with this issue I would greatly appreciate it.
如果有人能帮助我解决这个问题,我将不胜感激。 One idea that I thought might be the issue would have to deal with cookies but Im not experienced enough in web based applications to know if that is the case.
我认为可能是问题的一个想法必须处理 cookie,但我在基于 Web 的应用程序方面经验不足,无法知道是否是这种情况。
webpage = r"http://www.footlocker.com/"
driver = webdriver.Chrome(r'C:\Users\saleh\Downloads\chromedriver_win32\chromedriver.exe')
driver.get(webpage)
driver.find_elements_by_xpath("//*[contains(text(), 'Sitemap')]")[0].click()
Try simulate what a normal browser would do:尝试模拟普通浏览器会做什么:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
opts = Options()
# Add headers
user_agent = ('Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) '
'AppleWebKit/537.36 (KHTML, like Gecko) '
'Chrome/39.0.2171.95 Safari/537.36')
opts.add_argument(f'user-agent={user_agent}')
# Remove the Automation Info
opts.add_argument('--disable-infobars')
# if you move chromedriver.exe into C:\Windows or C:\Users\saleh or location where this code is executed, then you don’t have to pass it here
chrome_exe = r'C:\Users\saleh\Downloads\chromedriver_win32\chromedriver.exe'
driver = webdriver.Chrome(chrome_exe, chrome_options=opts)
URL should not repr r
r
webpage = 'http://www.footlocker.com/' driver.get(webpage)
Observer what you see.观察你所看到的。 Open Developer Tools, and perform the step manually first while observer the elements you are interacting.
打开开发人员工具,并首先手动执行该步骤,同时观察您正在交互的元素。 Then write code to do the same steps.
然后编写代码来做同样的步骤。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.