Python3 Asyncio从初始连接创建辅助连接

Question

I am trying to add two coroutines to asyncio loop and getting an error:

 RuntimeError: This event loop is already running 

My objective is to communicate to a server (that I have no control of). This server expects an initial connection from the client. The server then provided a port to the client on this connection. The client has to use this port to create a second connection. This second connection is used by the server to send unsolicited messages to the client. The first connection remains up throughout for other two-way communications.

To recreate this scenario, I have some code that reproduces the error:

class Connection():
    def __init__(self, ip, port, ioloop):
        self.ip = ip
        self.port = port
        self.ioloop = ioloop
        self.reader, self.writer = None, None
        self.protocol = None
        self.fileno = None

    async def __aenter__(self):
        # Applicable when doing 'with Connection(...'
        log.info("Entering and Creating Connection")
        self.reader, self.writer = (
            await asyncio.open_connection(self.ip, self.port, loop=self.ioloop)
        )
        self.protocol = self.writer.transport.get_protocol()
        self.fileno = self.writer.transport.get_extra_info('socket').fileno()

        log.info(f"Created connection {self}")
        return self

    async def __aexit__(self, *args):
        # Applicable when doing 'with Connection(...'
        log.info(f"Exiting and Destroying Connection {self}")
        if self.writer:
            self.writer.close()

    def __await__(self):
        # Applicable when doing 'await Connection(...'
        return self.__aenter__().__await__()

    def __repr__(self):
        return f"[Connection {self.ip}:{self.port}, {self.protocol}, fd={self.fileno}]"

    async def send_recv_message(self, message):
        log.debug(f"send: '{message}'")
        self.writer.write(message.encode())
        await self.writer.drain()

        log.debug("awaiting data...")
        data = await self.reader.read(9999)
        data = data.decode()
        log.debug(f"recv: '{data}'")
        return data


class ServerConnection(Connection):
    async def setup_connection(self):
        event_port = 8889  # Assume this came from the server
        print("In setup connection")
        event_connection = await EventConnection('127.0.0.1', event_port, self.ioloop)
        self.ioloop.run_until_complete(event_connection.recv_message())

class EventConnection(Connection):
    async def recv_message(self):
        log.debug("awaiting recv-only data...")
        data = await self.reader.read(9999)
        data = data.decode()
        log.debug(f"recv only: '{data}'")
        return data


async def main(loop):
    client1 = await ServerConnection('127.0.0.1', 8888, loop)
    await client1.setup_connection()
    await client1.send_recv_message("Hello1")
    await client1.send_recv_message("Hello2")
    await asyncio.sleep(5)

if __name__ == '__main__':
    #logging.basicConfig(level=logging.INFO)
    logging.basicConfig(level=logging.DEBUG)
    log = logging.getLogger()
    ioloop = asyncio.get_event_loop()
    print('starting loop')
    ioloop.run_until_complete(main(ioloop))
    print('completed loop')
    ioloop.close()

The error occurs in ServerConnection.setup_connection() method where run_until_complete is being called.

I am probably doing something wrong due to lack of understanding asyncio. Basically, how do I setup a secondary connection which will get event notifications (unsolicited) while setting up the first connection?

Thanks.

Followup

Since the code is very similar (a few changes to add more functionality to it), I hope it's not bad etiquette to followup to the original post as the resulting error is still the same.

The new issue is that when it receives the unsolicited message (which is received by EventConnection), the recv_message calls process_data method. I would like to make process_data be a future so that recv_message completes (ioloop should stop). The ensure_future would then pick it up and continue running again to use ServerConnection to do a request/response to the server. Before it does that though, it has to go to some user code (represented by external_command() and from whom I would prefer to hide the async stuff). This would make it synchronous again. Hence, once they've done what they need to, they should call execute_command on ServerConnection, which then kicks off the loop again.

The problem is, my expectation for using ensure_future didn't pan out as it seems the loop didn't stop from running. Hence, when the code execution reaches execute_command which does the run_until_complete, an exception with the error "This event loop is already running" occurs.

I have two questions:

1. How can I make it so that the ioloop can stop after process_data is placed into ensure_future, and subsequently be able to run it again in execute_command?

2. Once recv_message has received something, how can we make it so that it can receive more unsolicited data? Is it enough/safe to just use ensure_future to call itself again?

Here's the example code that simulates this issue.

client1 = None

class ServerConnection(Connection):
    connection_type = 'Server Connection'
    async def setup_connection(self):
        event_port = 8889  # Assume this came from the server
        print("In setup connection")
        event_connection = await EventConnection('127.0.0.1', event_port, self.ioloop)
        asyncio.ensure_future(event_connection.recv_message())

    async def _execute_command(self, data):
        return await self.send_recv_message(data)

    def execute_command(self, data):
        response_str = self.ioloop.run_until_complete(self._execute_command(data))
        print(f"exec cmd response_str: {response_str}")

    def external_command(self, data):
        self.execute_command(data)


class EventConnection(Connection):
    connection_type = 'Event Connection'
    async def recv_message(self):
        global client1
        log.debug("awaiting recv-only data...")
        data = await self.reader.read(9999)
        data = data.decode()
        log.debug(f"recv-only: '{data}'")
        asyncio.ensure_future(self.process_data(data))
        asyncio.ensure_future(self.recv_message())

    async def process_data(self, data):
        global client1
        await client1.external_command(data)


async def main(ioloop):
    global client1
    client1 = await ServerConnection('127.0.0.1', 8888, ioloop)
    await client1.setup_connection()
    print(f"after connection setup loop running is {ioloop.is_running()}")
    await client1.send_recv_message("Hello1")
    print(f"after Hello1 loop running is {ioloop.is_running()}")
    await client1.send_recv_message("Hello2")
    print(f"after Hello2 loop running is {ioloop.is_running()}")
    while True:
        print(f"inside while loop running is {ioloop.is_running()}")
        t = 10
        print(f"asyncio sleep {t} sec")
        await asyncio.sleep(t)


if __name__ == '__main__':
    logging.basicConfig(level=logging.DEBUG)
    log = logging.getLogger()
    ioloop = asyncio.get_event_loop()
    print('starting loop')
    ioloop.run_until_complete(main(ioloop))
    print('completed loop')
    ioloop.close()

Answer 1

Try replacing:

self.ioloop.run_until_complete

With

await