Python 3 urllib.request.urlopen

Question

How can I avoid exceptions from urllib.request.urlopen if response.status_code is not 200? Now it raise URLError or HTTPError based on request status.

Is there any other way to make request with python3 basic libs?

How can I get response headers if status_code != 200 ?

Answer 1

Use try except , the below code:

from urllib.request import Request, urlopen
from urllib.error import URLError, HTTPError
req = Request("http://www.111cn.net /")
try:
    response = urlopen(req)
except HTTPError as e:
    # do something
    print('Error code: ', e.code)
except URLError as e:
    # do something
    print('Reason: ', e.reason)
else:
    # do something
    print('good!')

Answer 2

The docs state that the exception type, HTTPError , can also be treated as a HTTPResponse . Thus, you can get the response body from an error response as follows:

import urllib.request
import urllib.error

def open_url(request):
  try:
    return urllib.request.urlopen(request)
  except urllib.error.HTTPError as e:
    # "e" can be treated as a http.client.HTTPResponse object
    return e

and then use as follows:

result = open_url('http://www.stackoverflow.com/404-file-not-found')
print(result.status)           # prints 404
print(result.read())           # prints page contents
print(result.headers.items())  # lists headers

Answer 3

I found a solution from py3 docs

>>> import http.client
>>> conn = http.client.HTTPConnection("www.python.org")
>>> # Example of an invalid request
>>> conn.request("GET", "/parrot.spam")
>>> r2 = conn.getresponse()
>>> print(r2.status, r2.reason)
404 Not Found
>>> data2 = r2.read()
>>> conn.close()

https://docs.python.org/3/library/http.client.html#examples