How to read from socket using asyncio add_reader

Question

I have this code:

import sys
import socket
import asyncio

async def main(dest_addr, max_hops=30, timeout=0.5):
    loop = asyncio.get_event_loop()
    queue = asyncio.Queue()

    port = 33434

    rx = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP)
    rx.settimeout(timeout)
    rx.bind(("", port))

    def reader():
        try:
            _, addr = rx.recvfrom(512)
            addr = addr[0]
        except socket.timeout:
            addr = None
        queue.put_nowait(addr)

    loop.add_reader(rx, reader)

    tx = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)

    for ttl in range(1, max_hops + 1):
        tx.setsockopt(socket.SOL_IP, socket.IP_TTL, ttl)
        tx.sendto(b"", (dest_addr, port))
        addr = await queue.get()
        print(ttl, addr)

    loop.remove_reader(rx)


if __name__ == "__main__":
    dest_name = sys.argv[1]
    dest_addr = socket.gethostbyname(dest_name)
    print(f"traceroute to {dest_name} ({dest_addr})")
    asyncio.get_event_loop().run_until_complete(main(dest_addr))

I am basically trying to implement traceroute using asyncio.

I'm monitoring the socket file descriptor for read availability and invoke reader when I receive data from a device after using the socket.sendto method and wait for the queue to be filled before going to the next step.

However, I my program hangs right after the first iteration, on the second addr = await queue.get() .

It seems like the reader callback is only invoked once and never again so the queue is not filled, which is strange because I have a timeout of 0.5s on the rx socket.

Answer 1

Answering my own question:

I think what happens is that, the device (my front router for example) doesn't respond anything so I am never notified when the file descriptor is ready for reading, so the callback is not invoked.

The workaround is to wrap the queue.get() within asyncio.wait_for with a timeout so it doesn't hang forever:

for ttl in range(1, max_hops + 1):
    tx.setsockopt(socket.SOL_IP, socket.IP_TTL, ttl)
    tx.sendto(b"", (dest_addr, port))
    try:
        addr = await asyncio.wait_for(queue.get(), timeout)
    except asyncio.TimeoutError:
        addr = "timeout"
    print(ttl, addr)