Python asyncio SSL transport not closing

Question

I am trying to build an RDP connection to a Windows 10 VM using python asyncio Protocol. After the first message exchange negotiating a TLS connection, one has to upgrade the connection to TLS, for which I use start_tls. Everything works fine until I try to close the connection after the second packet by calling transport.close(). The documentation says that the "connection_lost" method of the protocol instance is called with None after all output buffers are flushed ( Source ). In my case, close() is called and the call finishes, but connection_lost() is never called, which makes my application wait until the server sends an RST after a timeout, which triggers connection_lost correctly. This does happen if the server negotiates TLS 1.2, yet it does not for TLS 1.3. I have checked that the output buffer is empty and I looked through the cpython source but could not figure out where it hangs. With Wireshark I saw that nothing happens when close() is called, my client sends nothing to the server. I am using python 3.8.5 on Ubuntu 20.04.

Does anyone have an idea why connection_lost is not called?

See my minimum working example below:

#!/usr/bin/env python3

import asyncio
import ssl
import binascii

# first two packets for RDP, tested thoroughly
PACKET_NEGO = binascii.unhexlify("0300002d28e00000000000436f6f6b69653a206d737473686173683d6c696f6e6c656c0d0a0100080001000000")
PACKET_CONN = binascii.unhexlify("030001ca02f0807f658201be0401010401010101ff30200202002202020002020200000202000102020000020200010202ffff020200023020020200010202000102020001020200010202000002020001020204200202000230200202ffff0202fc170202ffff0202000102020000020200010202ffff020200020482014b000500147c00018142000800100001c00044756361813401c0d800040008000004000301ca03aa09040000280a00006e006f0076006f007300690062006900720073006b000000000000000000000004000000000000000c0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001ca01000000000018000b0001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000002c00c00000000000000000003c0440005000000636c697072647200c0a00000726470736e640000c0000000736e646462670000c0000000726470647200000080800000647264796e766300c000000004c00c000d00000000000000")

class RDPConnection(asyncio.Protocol):
    def __init__(self, on_con_close, on_tls, loop, ip):
        self.on_tls = on_tls
        self.on_con_close = on_con_close
        self.loop = loop
        self.state = 0
        self.ip = ip

    def connection_made(self, transport):
        self.transport = transport

        if self.state == 0:
            transport.write(PACKET_NEGO) # write first packet on initial connection
        else:
            transport.write(PACKET_CONN) # write second packet on TLS connection

    def data_received(self, data):
        if self.state == 0:
            self.state += 1
            self.on_tls.set_result(True) # start TLS connection
        else:
            print("closing connection")
            self.transport.close()
            print("closed connection")
        
    def connection_lost(self, exc):
        print("connection lost called")
        self.on_con_close.set_result(True)
        print("{}: {}".format(self.ip, exc))

    def _timeout(self):
        if self.transport:
            self.transport.close()
        
async def handle_connection(loop, ip, timeout):
    on_tls = loop.create_future() # called when connection is ready to be upgraded to TLS
    con_close = loop.create_future() # called when connection is closed

    try:
        transport, protocol = await asyncio.wait_for(loop.create_connection(lambda: RDPConnection(con_close, on_tls, loop, ip), ip, 3389), timeout=timeout)
    except:
        return
    await on_tls # wait for first response

    ssl_ctx = ssl._create_unverified_context() # VM uses self signed certificate
    try:
        transport2 = await asyncio.wait_for(loop.start_tls(transport, protocol, ssl_ctx), timeout=timeout) # upgrade to TLS
        protocol.connection_made(transport2) # set TLS transport
    except:
        return
    await con_close # wait for connection_lost to be called
    return

async def main():
    loop = asyncio.get_running_loop()
    await handle_connection(loop, "my VM", 10)

asyncio.run(main())

Output on my machine:

closing connection
closed connection
connection lost called
my VM: [Errno 104] Connection reset by peer

Please note that "connection lost called" is only called when the connection is reset by the server (after ~30 seconds)

Answer 1

transport.abort() is your friend.

reference