python - Why does TCP client slow down then produce OSError: [Errno 99] Cannot assign requested address

Question

I am writing a server for an event-based simulator and am using asyncio TCP server for this purpose.

#server.py
import asyncio
import itertools
import json


class Server:
    def __init__(self, loop=None):
        self.loop = loop
        self.pq = asyncio.PriorityQueue()
        self.counter = itertools.count()

    async def __call__(self, reader, writer):
        event = await reader.read(100)
        message = json.loads(event.decode())
        self.pq.put_nowait([next(self.counter), message])
        while self.pq.qsize():
            t = await self.pq.get()
            send_data = json.dumps(t).encode("utf-8")
            writer.write(send_data)
            await writer.drain()

loop = asyncio.get_event_loop()
s = Server(loop)
coro = asyncio.start_server(s, '127.0.0.1', 5000, loop=loop)
server = loop.run_until_complete(coro)

# Serve requests until Ctrl+C is pressed
print('Serving on {}'.format(server.sockets[0].getsockname()))
try:
    loop.run_forever()
except KeyboardInterrupt:
    pass

# Close the server
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()

I want a client to send encoded json events to this server quickly.

#client.py
import socket
import json
import datetime

host = "127.0.0.1"
port = 5000

N = 10000
start = datetime.datetime.utcnow()


for i in range(1, N + 1):
    s = socket.create_connection((host, port))
    send_message = {"id": i, "value": i * 3}
    send_json = json.dumps(send_message)
    send_data = send_json.encode("utf-8")
    s.sendall(send_data)
    receive_data = s.recv(1024)
    receive_json = receive_data.decode("utf-8")
    _ = json.loads(receive_json)
    s.close()

stop = datetime.datetime.utcnow()
print("Tasks per second: {}".format(N / (stop - start).total_seconds()))

Problem The client program has varying performance and error generation, despite having no other user programs open.

Often, but not always, the first run of client.py runs at about 3,000 tasks per second. Occasionally, the first run goes slower (~500-600 tasks per second).

Once the performance drops to 500-600 tasks per second, further runs never recover back to 3,000 tasks per second.

Eventually, running client.py raises the following exception:

Traceback (most recent call last):
  File "aioclient.py", line 12, in <module>
    s = socket.create_connection((host, port))
  File "/home/randm/Libraries/anaconda3/lib/python3.6/socket.py", line 724, in create_connection
    raise err
  File "/home/randm/Libraries/anaconda3/lib/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
OSError: [Errno 99] Cannot assign requested address

Question

How should I rewrite client.py (or server.py ) to avoid this?

Having read https://docs.python.org/3/howto/sockets.html , perhaps a few notes:

1. The messages will be variable length.
2. I am fine with with either delimiting or sending message length with the messages as opposed to shutting down the connection, which seems to be the underlying source of the problem here.
3. I'd rather not add an application-layer protocol like HTTP here as I know messages will always be UTF8-encoded JSON.

Answer 1

I think using a message delimiter (I've chosen b'\\x1e' ) allows me to have 1 connection for the entire set of messages instead of establishing a new connection for each message. The StreamReader.readuntil method works fine in this case.

# server.py
import asyncio
import itertools
import json


PQ = asyncio.PriorityQueue()
COUNTER = itertools.count()


async def handle_data_provider(reader, writer):
    try:
        while True:
            data = await reader.readuntil(b'\x1e')
            message = json.loads(data[:-1].decode())
            n = next(COUNTER)
            if n % 10000 == 0:
                print(n, PQ.qsize())
            PQ.put_nowait([n, message])
    except asyncio.streams.IncompleteReadError:
        pass

loop = asyncio.get_event_loop()
coro = asyncio.start_server(handle_data_provider, '127.0.0.1', 5000, loop=loop)
server = loop.run_until_complete(coro)

# Serve requests until Ctrl+C is pressed
print('Serving on {}'.format(server.sockets[0].getsockname()))
try:
    loop.run_forever()
except KeyboardInterrupt:
    pass

# Close the server
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()

And client...

# client.py
import socket
import json
import datetime

host = "127.0.0.1"
port = 5000

N = 10000
start = datetime.datetime.utcnow()

s = socket.create_connection((host, port))

for i in range(1, N + 1):
    message = {"id": i, "value": i * 3}
    json_message = json.dumps(message)
    data = json_message.encode("utf-8") + b'\x1e'
    s.sendall(data)

s.close()

stop = datetime.datetime.utcnow()
print("Tasks per second: {}".format(N / (stop - start).total_seconds()))