Python not resolving host (timed out) when running with Gunicorn inside a Docker container (weasyprint needs to do that)

Question

I have a Django app running in a docker container, which serves the app using Gunicorn. This app uses Weasyprint to generate a PDF, which loads a CSS dynamically. This CSS is served by the app, and it works fine (and loads instantly): host_pointing_to_that_same_machine.com/dynamic_css_path.css

(it's NOT a static file, it's actually a path defined in the django URLs that outputs CSS)

That's the weasyprint code:

css = weasyprint.CSS(url=request.build_absolute_uri(reverse('dynamic_css_path')))
        response = HttpResponse(pdf.write_pdf(stylesheets=[css]), content_type='application/pdf')

This creates that URL correctly, and everything works good if instead of gunicorn I use the django built-in server (python manage.py runserver), both in my development machine and at the remote server inside the docker and so.

But when I serve the app via gunicorn: gunicorn project.wsgi:application --bind 0.0.0.0:8000

Then python is unable to resolve the host name, which I proved by doing:

1) Entering the docker's app console and trying it:

docker exec -it container_name /bin/ash
# ping host_pointing_to_that_same_machine.com
# ping any_other_host_pointing_an_external_machine.com

Everything is fine.

2) Entering Django's console inside the docker container:

# python manage.py shell
>>> import urllib
>>> r = urllib.request.urlopen(url='http://host_pointing_to_that_same_machine.com', timeout=10)
>>> print('status: '+str(r.status))

Returns 200; everything OK.

3) Entering python's shell inside the docker container:

# python
>>> from urllib import request
>>> r = request.urlopen(url='http://host_pointing_to_that_same_machine.com', timeout=10)
>>> print('status: '+str(r.status))
status: 200

4) Executing these lines of code inside the script, having Gunicorn serving them:

# docker logs container_name
Internal Server Error: /admin/app_name/path_to_pdf_generating_script/
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py", line 35, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 128, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 126, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/utils/decorators.py", line 142, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/contrib/admin/sites.py", line 223, in inner
    return view(request, *args, **kwargs)
  File "/srv/apps/project/admin/PdfMakingAdmin.py", line 125, in method_called_by_the_view
    timeout=10)
  File "/usr/local/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/local/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/local/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python3.6/urllib/request.py", line 1346, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/local/lib/python3.6/urllib/request.py", line 1321, in do_open
    r = h.getresponse()
  File "/usr/local/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/local/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.6/http/client.py", line 258, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

But if I change the code and instead of host_pointing_to_that_same_machine.com tries to load any domain that is not pointing to that machine, like http://example.com or anything:

# docker logs container_name
status: 200

I'm going for any of the other thousand ways I have to achieve the results I want, but still, this is bugging me and I wonder if this question and potential anwers are going to be useful for other people facing that, wether they are trying to use Weasyprint or any other script that needs to make http queries.

I'm leaving here the full traceback that weasyprint throws when trying the original code:

Environment:


Request Method: GET
Request URL: http://host_pointing_to_that_same_machine.com/admin/path-to-app/

Django Version: 2.0
Python Version: 3.6.8

Traceback:

File "/usr/local/lib/python3.6/site-packages/weasyprint/urls.py" in fetch
  284.         result = url_fetcher(url)

File "/usr/local/lib/python3.6/site-packages/weasyprint/urls.py" in default_url_fetcher
  248.                            timeout=timeout, context=ssl_context)

File "/usr/local/lib/python3.6/urllib/request.py" in urlopen
  223.     return opener.open(url, data, timeout)

File "/usr/local/lib/python3.6/urllib/request.py" in open
  526.         response = self._open(req, data)

File "/usr/local/lib/python3.6/urllib/request.py" in _open
  544.                                   '_open', req)

File "/usr/local/lib/python3.6/urllib/request.py" in _call_chain
  504.             result = func(*args)

File "/usr/local/lib/python3.6/urllib/request.py" in http_open
  1346.         return self.do_open(http.client.HTTPConnection, req)

File "/usr/local/lib/python3.6/urllib/request.py" in do_open
  1321.             r = h.getresponse()

File "/usr/local/lib/python3.6/http/client.py" in getresponse
  1331.                 response.begin()

File "/usr/local/lib/python3.6/http/client.py" in begin
  297.             version, status, reason = self._read_status()

File "/usr/local/lib/python3.6/http/client.py" in _read_status
  258.         line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")

File "/usr/local/lib/python3.6/socket.py" in readinto
  586.                 return self._sock.recv_into(b)

During handling of the above exception (timed out), another exception occurred:

File "/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
  35.             response = get_response(request)

File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  128.                 response = self.process_exception_by_middleware(e, request)

File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  126.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/usr/local/lib/python3.6/site-packages/django/utils/decorators.py" in _wrapped_view
  142.                     response = view_func(request, *args, **kwargs)

File "/usr/local/lib/python3.6/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  44.         response = view_func(request, *args, **kwargs)

File "/usr/local/lib/python3.6/site-packages/django/contrib/admin/sites.py" in inner
  223.             return view(request, *args, **kwargs)

File "/srv/apps/app_name/admin/PdfAdmin.py" in attendee_list
  134.         css = weasyprint.CSS(url=request.build_absolute_uri(reverse('view_to_dynamic_css')))

File "/usr/local/lib/python3.6/site-packages/weasyprint/__init__.py" in __init__
  322.         with result as (source_type, source, base_url, protocol_encoding):

File "/usr/local/lib/python3.6/contextlib.py" in __enter__
  81.             return next(self.gen)

File "/usr/local/lib/python3.6/site-packages/weasyprint/__init__.py" in _select_source
  407.         with fetch(url_fetcher, url) as result:

File "/usr/local/lib/python3.6/contextlib.py" in __enter__
  81.             return next(self.gen)

File "/usr/local/lib/python3.6/site-packages/weasyprint/urls.py" in fetch
  286.         raise URLFetchingError('%s: %s' % (type(exc).__name__, str(exc)))

Exception Type: URLFetchingError at /admin/app_name/path-to-pdf-generator/
Exception Value: timeout: timed out

Answer 1

在示例中，您正在访问http://host_pointing_to_that_same_machine.com，但在错误日志中，我看到您正在尝试访问http://host_pointing_to_that_same_machine.com/admin/path-to-app/

Answer 2

In case of host pointing to the same machine you needed to resolve the host by making a host entry into your \etc\hosts file. For docker container you can do that by including below configuration into your composer file:

extra_hosts:
  - "host_pointing_to_that_same_machine.com:192.168.X.X"

Replace the ip with your domain pointing ip. Note: If there multiple container u might be needed to include the extra_hosts for others as well.

Answer 3

there was a same thing for me, solution was that i've runned django via gunicorn only with one worker, like that:

  python manage.py migrate && 
gunicorn your_project.wsgi:application --bind 0.0.0.0:8888

the solution is to increase gunigorn workers from 1 to 2 ) with option --workers like that:

python manage.py migrate &&
gunicorn your_project.wsgi:application --workers=3 --bind 0.0.0.0:8888