如果需要使用硒登录，如何抓取网站？
[英]How scrape website if login is required using selenium?
问题描述
I am trying to scrape LinkedIn website for some user profiles. 
我正尝试在LinkedIn网站上抓取一些用户个人资料。
I am using selenium for browser automation. 我正在使用硒来实现浏览器自动化。
I need all the profiles under https://www.linkedin.com/search/results/all/?keywords=director%20supply%20chain&origin=GLOBAL_SEARCH_HEADER&page=1 我需要https://www.linkedin.com/search/results/all/?keywords=director%20supply%20chain&origin=GLOBAL_SEARCH_HEADER&page=1下的所有配置文件
But the site asks for login details. 但是该网站要求登录详细信息。
How should i give my login details in the code? 我应该如何在代码中提供我的登录详细信息？
2 个解决方案
解决方案1
0 2018-11-17 02:59:49
You are going to need to find the username and password boxes and enter them in. I find this easiest to do with css element ids. 您将需要找到用户名和密码框，然后输入它们。我发现使用CSS元素ID最简单。 Selenium has a find_element_by_id method. 硒具有find_element_by_id方法。 Check out this little selenium auto login project I made: https://github.com/bnorquist/auto_login/blob/master/scripts/login.py#L7 看看我做的这个小硒自动登录项目： https : //github.com/bnorquist/auto_login/blob/master/scripts/login.py#L7
解决方案2
0 2018-11-20 13:41:56
This code is working 该代码有效
driver.get("https://www.linkedin.com")
driver.implicitly_wait(6)
driver.find_element_by_xpath("""//*[@id="login-email"]""").send_keys(userid)
driver.find_element_by_xpath("""//*[@id="login-password"]""").send_keys(password)
driver.find_element_by_xpath("""//*[@id="login-submit"]""").click()
driver.get("https://www.linkedin.com/search/results/all/?
keywords=director%20supply%20chain&origin=GLOBAL_SEARCH_HEADER&page=1")
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.