[英]Web scraping requests python
Good morning everyone, I'm trying to access the html code of a web page, I've got the login working through python but only that once I print the page the html code is not as complete as if I opened it in a browser... Good morning everyone, I'm trying to access the html code of a web page, I've got the login working through python but only that once I print the page the html code is not as complete as if I opened it in a browser ...
loginurl = ('https://sgv.ivu-cloud.com/mbweb/j_security_check')
secure_url = ('https://sgv.ivu-cloud.com/mbweb/main/matter/desktop/main-menu')
session = HTMLSession()
payload = {
'j_username' : creds.username,
'j_password': creds.password
}
s = requests.Session()
s.post(loginurl, data=payload)
r = s.get((secure_url), cookies={'from-my': 'browser'})
soup = BeautifulSoup(r.content, 'html.parser')
c = session.get('https://sgv.ivu-cloud.com/mbweb/main/matter/desktop/main-menu#duty-details?beginDate=2021-12-14&allocatedEmployeeId=22480')
d = s.get('https://sgv.ivu-cloud.com/mbweb/main/matter/desktop/main-menu#duty-details?beginDate=2021-12-14&allocatedEmployeeId=22480')
print(d)
a this is the answer:这是答案:
IVU.plan Portal var mbwebBaseUrl = '/mbweb';600 600$(function () { jQuery.struts2_jquery.version = "4.0.3"; jQuery.struts2_jquery.loadAtOnce = true; jQuery.scriptPath = "/mbweb/struts/"; jQuery.struts2_jquery.local = "de"; jQuery.struts2_jquery.gridLocal = "de"; jQuery.struts2_jquery.timeLocal = "de"; jQuery.struts2_jquery.datatablesLocal = "de"; jQuery.ajaxSettings.traditional = true; jQuery.ajaxSetup({ cache: false }); jQuery.struts2_jquery.require("js/struts2/jquery.ui.struts2.min.js?s2j=4.0.3"); });<script type="text/javascript" src="/mbweb/struts/js/base/jquery-2.2.4.min.js"></script> <script type="text/javascript" src="/mbweb/struts/js/base/jquery-ui.min.js?s2j=4.0.3"></script> <script type="text/javascript" src="/mbweb/struts/i18n/datepicker-de.min.js?s2j=4.0.3"></script>
<link id="jquery_theme_link" rel="stylesheet" href="/mbweb/struts/themes/smoothness/jquery-ui.css?s2j=4.0.3" type="text/css"/>
you can use your code to login and get the cookies.您可以使用您的代码登录并获取 cookies。 then you can pass the cookies to selenium and continue然后您可以将 cookies 传递给 selenium 并继续
from selenium import webdriver
driver = webdriver.Firefox(executable_path="driver/geckodriver.exe")
for c in s.cookies:
driver.add_cookie({'name': c.name, 'value': c.value})
driver.get('https://sgv.ivu-cloud.com/mbweb/main/matter/desktop/main-menu#duty-details?beginDate=2021-12-14&allocatedEmployeeId=22480')
this will only work if you have logged-in first using your code, so this needs to be added to your code in the end这仅在您首先使用您的代码登录时才有效,因此最后需要将其添加到您的代码中
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.