How to handle DNS timeouts with aiohttp?

Question

The aiohttp readme says:

If you want to use timeouts for aiohttp client please use standard asyncio approach: yield from asyncio.wait_for(client.get(url), 10)

But that doesn't handle DNS timeouts which are, I guess, handled by the OS. Also the with aiohttp.Timeout doesn't handle OS DNS lookups.

There has been a discussion at the asyncio repo without final conclusion and Saghul has made aiodns but I'm not sure how to mix it into aiohttp and whether that will allow asyncio.wait_for functionality.

Testcase (takes 20 sec on my linux box):

async def fetch(url):
    url = 'http://alicebluejewelers.com/'
    with aiohttp.Timeout(0.001):
        resp = await aiohttp.get(url)

Answer 1

Timeout works as expected but unfortunately your example hangs on python shutdown procedure: it waits for termination of background thread which performs DNS lookup.

As a solution I can suggest using aiodns for manual IP resolving:

import asyncio
import aiohttp
import aiodns

async def fetch():
    dns = 'alicebluejewelers.com'
    # dns = 'google.com'
    with aiohttp.Timeout(1):
        ips = await resolver.query(dns, 'A')
        print(ips)
        url = 'http://{}/'.format(ips[0].host)
        async with aiohttp.get(url) as resp:
            print(resp.status)

loop = asyncio.get_event_loop()
resolver = aiodns.DNSResolver(loop=loop)
loop.run_until_complete(fetch())

Maybe solution worth to be included into TCPConnector as optional feature.

Pull Request is welcome!