python - requests.get（连接中止。'，OSError（“（60，'ETIMEDOUT'）

Question

I'm new to web scraper, tried to access a website but got too many errors and was told (Connection aborted.', OSError("(60, 'ETIMEDOUT').

from bs4 import BeautifulSoup
import requests
urla="https://www.loopnet.com/search/office-space/san-diego-ca/for-lease/?sk=02fa1ad85634ef43bfd21f24bbe3a14a"
source = requests.get(urla).text
print(source)

---

OSError Traceback (most recent call last) ~/opt/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw) 671 headers=headers, --> 672 chunked=chunked, 1343 try: -> 1344 response.begin() 1345 except ConnectionError:

~/opt/anaconda3/lib/python3.7/http/client.py in begin(self) 305 while True: --> 306 version, status, reason = self._read_status() 307 if status:= CONTINUE:

~/opt/anaconda3/lib/python3.7/http/client.py in _read_status(self) 266 def _read_status(self): --> 267 line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") 268 if len(line) > _MAXLINE:

~/opt/anaconda3/lib/python3.7/socket.py in readinto(self, b) 588 try: --> 589 return self._sock.recv_into(b) 590 except timeout:

~/opt/anaconda3/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py in recv_into(self, *args, **kwargs) 317 else: --> 318 raise SocketError(str(e)) 319 except OpenSSL.SSL.ZeroReturnError:

OSError: (60, 'ETIMEDOUT')

During handling of the above exception, another exception occurred:

ProtocolError Traceback (most recent call last) ~/opt/anaconda3/lib/python3.7/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies) 448 retries=self.max_retries, --> 449 timeout=timeout 450 )

~/opt/anaconda3/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw) 719 retries = retries.increment( --> 720 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] 721 )

ConnectionError: ('Connection aborted.', OSError("(60, 'ETIMEDOUT')")) (Something like this)

Answer 1

The reason you are getting this error is because the server is not responding or is taking too long to respond. you should always encapsulate the requests in a try block to avoid program crash, This way you can catch any error, or specific errors by using the requests.exceptions module

import requests
url="https://www.loopnet.com/search/office-space/san-diego-ca/for-lease/?sk=02fa1ad85634ef43bfd21f24bbe3a14a"

Catch any error

try:
    source = requests.get(url).text
except:
    print 'ERROR'

Catch ConnectionError

from requests.exceptions import ConnectionError

try:
    source = requests.get(url).text
except ConnectionError:                  # <-- this is your case scenario
    print 'NOT RESPONDING'

Catch TimeoutError

from requests.exceptions import ReadTimeout

try:
    source = requests.get(url, timeout=2).text  # <-- you should always use timeout to avoid requests hanging or taking too long to respond
except ReadTimeout:
    print 'TIMEOUT'

You can view all the requests exceptions like this

for exception in dir(requests.exceptions):
    print exception

Read requests exceptions docs here: https://requests.readthedocs.io/en/latest/user/quickstart/#errors-and-exceptions