请求和 BeautifulSoup4 的连接错误。 昨晚我的代码有效，我没有改变任何东西

Question

我正在编写执行 web 抓取的代码。 我试图从剑桥词典网站获取 HTML 代码，但弹出一条错误消息。 如果您能告诉我错误的原因和解决此问题的方法，我将不胜感激。

这是我的代码：

import requests
from bs4 import BeautifulSoup
    
    



def checkWord(word):
    url_top = "https://dictionary.cambridge.org/dictionary/english/"
    url = url_top + word

    headers = requests.utils.default_headers()

    headers.update(
        {
            'User-Agent': 'My User Agent 1.0',
        }       
    )

    html = requests.get(url, headers=headers).text 
    soup = BeautifulSoup(html, 'html.parser') 
    check = soup.find("title")
    boolean = check.string

    
    if boolean == "Cambridge English Dictionary: Meanings & Definitions":
        return False
    else:
        return True

word = "App"
checkWord(word)

但是，错误发生在html = requests.get(url, headers=headers).text

错误信息如下所示——

Exception has occurred: ConnectionError
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

  File "<string>", line 3, in raise_from

During handling of the above exception, another exception occurred:

  File "<string>", line 3, in raise_from

During handling of the above exception, another exception occurred:

Answer 1

您的代码一直运行良好。 最有可能的问题是你的本地 inte.net 这就是为什么它可能是暂时的或检查你的 inte.net 连接

import requests
from bs4 import BeautifulSoup
    
def checkWord(word):
    url_top = "https://dictionary.cambridge.org/dictionary/english/"
    url = url_top + word

    headers = requests.utils.default_headers()

    headers.update(
        {
            'User-Agent': 'Mozilla/5.0',
        }       
    )

    html = requests.get(url, headers=headers).text 
    soup = BeautifulSoup(html, 'html.parser') 
    check = soup.find("title").text
    print(check)


word = "App"
checkWord(word)

Output：

APP | meaning, definition in Cambridge English Dictionary

Answer 2

看起来远程主机禁止了您。 如果您仍然可以使用 web 浏览器从您的计算机打开该网站，请尝试将用户代理更改为如下所示：

"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"