python网络搜寻器，当我搜寻URL时，status_code显示405

Question

最近，我正在学习python。 当我使用BeautifulSoup并请求获取html时，我的状态是405。而且，汤是错误的。 我访问了URL 。

这是我的代码：

def craw(url):
    user_agent = 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:55.0) Gecko/20100101 Firefox/55.0'
    headers = {'User_Agent':user_agent,
'Accept':'*/*',
'Accept-Encoding':'gzip, deflate',
'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',

'Accept-Language':'zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3',
'Host':'www.qichacha.com',
'Referer':'http://www.qichacha.com/search?key=%E5%A9%9A%E5%BA%86',
}
    response = requests.get(url,headers = headers)
    if response.status_code != 200:
        response.encoding = 'utf-8'
        print(response.status_code)
        print('ERROR')

    soup = BeautifulSoup(response.text,'lxml')
    print(soup)
if __name__ == '__main__':
url = r'http://www.qichacha.com/search?key=%E5%A9%9A%E5%BA%86'
s1 = craw(url)

输出：

 405 ERROR <!DOCTYPE html> <html lang="zh-cn"> <head> <meta charset="utf-8"/> <meta content="IE=edge,chrome=1" http-equiv="X-UA-Compatible"/> <meta content="a3c0e" name="data-spm"/> <title>405</title> <style> html, body, div, a, h2, p { margin: 0; padding: 0; font-family: 微软雅黑; } a { text-decoration: none; color: #3b6ea3; } .container { width: 1000px; margin: auto; color: #696969; } .header { padding: 50px 0; } .header .message { height: 36px; padding-left: 120px; background: ur l(https://errors.aliyun.com/images/TB1TpamHpXXXXaJXXXXeB7nYVXX-104-162.png) no-r epeat 0 -128px; line-height: 36px; } .main { padding: 50px 0; background: #f4f5f7; } .main img { position: relative; left: 120px; } .footer { margin-top: 30px; text-align: right; } .footer a { padding: 8px 30px; border-radius: 10px; border: 1px soli d #4babec; } .footer a:hover { opacity: .8; } .alert-shadow { display: none; position: absolute; top: 0; left: 0; width: 100%; height: 100%; background: #999; opacity: .5; } .alert { display: none; position: absolute; top: 200px; left: 50%; w idth: 600px; margin-left: -300px; padding-bottom: 25px; border: 1px solid #ddd; box-shadow: 0 2px 2px 1px rgba(0, 0, 0, .1); background: #fff; font-size: 14px; color: #696969; } .alert h2 { margin: 0 2px; padding: 10px 15px 5px 15px; font-size: 14px; font-weight: normal; border-bottom: 1px solid #ddd; } .alert a { display: block; position: absolute; right: 10px; top: 8px ; width: 30px; height: 20px; text-align: center; } .alert p { padding: 20px 15px; } </style> </head> <body data-spm="7663354"> <div data-spm="1998410538"> <div class="header"> <div class="container"> <div class="message">很抱歉，由于您访问的URL有可能对网站造成安全威胁，您的访问被阻断。</div> </div> </div> <div class="main"> <div class="container"> <img src="https://errors.aliyun.com/images/TB15QGaHpXXXXXOaXXXXia39XXX-660-117.p ng"/> </div> </div> <div class="footer"> <div class="container"> <a data-spm-click="gostr=/waf.123.123;locaid=d001;" href="javascript:;" id="repo rt" target="_blank">误报反馈</a> </div> </div> </div> <div class="alert-shadow" id="alertShadow"></div> <div class="alert" id="alertContainer"> <h2>提示：<a href="javascript:;" id="closeAlert" title="关闭">X</a></h2> <p>感谢您的反馈，应用防火墙会尽快进行分析和确认。</p> </div> <script> function show() { var g = function(ele) { return document.getElementById(ele); }; var reportHandle = g('report'); var alertShadow = g('alertShadow'); var alertContainer = g('alertContainer'); var closeAlert = g('closeAlert'); var own = {}; own.report = function() { // SPM own.alert(); }; own.alert = function() { alertShadow.style.display = 'block'; alertContainer.style.display = 'block'; }; own.close = function() { alertShadow.style.display = 'none'; alertContainer.style.display = 'none'; }; }; </script> <script charset="utf-8" src="https://errors.aliyun.com/error.js?s=3" type="text/ javascript"></script> </body> </html> ------------------ (program exited with code: 0)请按任意键继续. . . 

根据我的输出，我知道我得到的汤不是我想要的页面。 但是问题出在哪里呢？ 我是新手。

Answer 1

我尝试解决这个问题。幸运的是，我已经使用完整的标头解决了该问题。

 headers = {
'Host':'www.qichacha.com',
'User-Agent':r'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:55.0) Gecko/20100101 
Firefox/55.0',
'Accept':'*/*',
'Accept-Language':'zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3',
'Accept-Encoding':'gzip, deflate',
'Referer':'http://www.qichacha.com/',
'Cookie':'use your cookies'
'If-Modified-Since':'Wed, 30 Aug 2017 10:48:38 GMT',
'If-None-Match':'"59a69806-80"',
'Cache-Control':'max-age=0',}