python-為什么TCP客戶端速度變慢然后產生OSError：[Errno 99]無法分配請求的地址

Question

我正在為基於事件的模擬器編寫服務器，並且為此目的使用了asyncio TCP服務器。

#server.py
import asyncio
import itertools
import json


class Server:
    def __init__(self, loop=None):
        self.loop = loop
        self.pq = asyncio.PriorityQueue()
        self.counter = itertools.count()

    async def __call__(self, reader, writer):
        event = await reader.read(100)
        message = json.loads(event.decode())
        self.pq.put_nowait([next(self.counter), message])
        while self.pq.qsize():
            t = await self.pq.get()
            send_data = json.dumps(t).encode("utf-8")
            writer.write(send_data)
            await writer.drain()

loop = asyncio.get_event_loop()
s = Server(loop)
coro = asyncio.start_server(s, '127.0.0.1', 5000, loop=loop)
server = loop.run_until_complete(coro)

# Serve requests until Ctrl+C is pressed
print('Serving on {}'.format(server.sockets[0].getsockname()))
try:
    loop.run_forever()
except KeyboardInterrupt:
    pass

# Close the server
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()

我希望客戶端將編碼的json事件快速發送到此服務器。

#client.py
import socket
import json
import datetime

host = "127.0.0.1"
port = 5000

N = 10000
start = datetime.datetime.utcnow()


for i in range(1, N + 1):
    s = socket.create_connection((host, port))
    send_message = {"id": i, "value": i * 3}
    send_json = json.dumps(send_message)
    send_data = send_json.encode("utf-8")
    s.sendall(send_data)
    receive_data = s.recv(1024)
    receive_json = receive_data.decode("utf-8")
    _ = json.loads(receive_json)
    s.close()

stop = datetime.datetime.utcnow()
print("Tasks per second: {}".format(N / (stop - start).total_seconds()))

問題盡管沒有打開其他用戶程序，該客戶端程序仍具有不同的性能和錯誤生成。

通常，但並非總是如此，第一次運行client.py速度約為每秒3,000個任務。 有時，第一次運行會變慢（每秒約500-600個任務）。

一旦性能下降到每秒500-600個任務，進一步的運行就永遠不會恢復到每秒3,000個任務。

最終，運行client.py會引發以下異常：

Traceback (most recent call last):
  File "aioclient.py", line 12, in <module>
    s = socket.create_connection((host, port))
  File "/home/randm/Libraries/anaconda3/lib/python3.6/socket.py", line 724, in create_connection
    raise err
  File "/home/randm/Libraries/anaconda3/lib/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
OSError: [Errno 99] Cannot assign requested address

題

我應該如何重寫client.py （或server.py ）以避免這種情況？

閱讀了https://docs.python.org/3/howto/sockets.html ，也許有幾點注意事項：

1. 消息將是可變長度的。
2. 我可以用消息來界定或發送消息長度，而不是關閉連接，這似乎是這里問題的根本根源。
3. 我寧願不在此處添加像HTTP這樣的應用程序層協議，因為我知道消息將始終是UTF8編碼的JSON。

Answer 1

我認為使用消息定界符（我選擇b'\\x1e' ）使我可以為整個消息集建立1個連接，而不是為每個消息建立新的連接。 在這種情況下， StreamReader.readuntil方法可以正常工作。

# server.py
import asyncio
import itertools
import json


PQ = asyncio.PriorityQueue()
COUNTER = itertools.count()


async def handle_data_provider(reader, writer):
    try:
        while True:
            data = await reader.readuntil(b'\x1e')
            message = json.loads(data[:-1].decode())
            n = next(COUNTER)
            if n % 10000 == 0:
                print(n, PQ.qsize())
            PQ.put_nowait([n, message])
    except asyncio.streams.IncompleteReadError:
        pass

loop = asyncio.get_event_loop()
coro = asyncio.start_server(handle_data_provider, '127.0.0.1', 5000, loop=loop)
server = loop.run_until_complete(coro)

# Serve requests until Ctrl+C is pressed
print('Serving on {}'.format(server.sockets[0].getsockname()))
try:
    loop.run_forever()
except KeyboardInterrupt:
    pass

# Close the server
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()

還有客戶

# client.py
import socket
import json
import datetime

host = "127.0.0.1"
port = 5000

N = 10000
start = datetime.datetime.utcnow()

s = socket.create_connection((host, port))

for i in range(1, N + 1):
    message = {"id": i, "value": i * 3}
    json_message = json.dumps(message)
    data = json_message.encode("utf-8") + b'\x1e'
    s.sendall(data)

s.close()

stop = datetime.datetime.utcnow()
print("Tasks per second: {}".format(N / (stop - start).total_seconds()))