Python slow at accepting connections via sockets

Question

So first I'll describe what I am doing.

A game is providing a webinterface but only on IPv4 and I would like people out in the internet to reach it too. From my ISP I however only get public IPv6. And since I couldn't find anything on the internet to translate requests and responses I wrote a little app that does. So IPv6 requests get forwarded to the webserver and the webservers responses get translated back to IPv6. That's working fine. The only troubling bit is that... not all requests get detected or whatever is happening, like I first visit the webpage and sometimes it just hangs and says that it's waiting for a style.css file but when I look at the console output there's no reported connection. And generally there's just a whole lot of delay when you try doing something with the webinterface.

Then here is my code: (A word of warning I don't really know what everything with the networking exactly does, the stuff around the sending I especially don't quite understand if it's even needed, I just found it online)

def handle_request(return_address):
    ipv4side = socket.create_connection(("127.0.0.1", 7245))
    request = return_address.recv(2048)
    print(request)
    request = str(request, 'utf-8')
    p = re.compile('\\[[^]]*]:7250')
    m = p.search(request)
    request = request.replace(request[m.start():m.end()], '127.0.0.1:7245')
    request = request.encode('utf-8')

    msg_len = len(request)
    totalsent = 0
    while totalsent < msg_len:
        sent = ipv4side.send(request[totalsent:])
        if sent == 0:
            raise RuntimeError("socket connection broken")
        totalsent += sent
    while True:
        response = ipv4side.recv(2048)

        if len(response) == 0:
            ipv4side.close()
            return

        msg_len = len(response)
        totalsent = 0
        while totalsent < msg_len:
            sent = return_address.send(response[totalsent:])
            if sent == 0:
                raise RuntimeError("socket connection broken")
            totalsent += sent


ipv6side = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
ipv6side.bind((IPV6, 7250))
ipv6side.listen(20)
ipv6side.settimeout(30)

while True:
    try:
        connected_socket = ipv6side.accept()[0]
        print("NEW CONNECTION!" + str(connected_socket))
        Thread(target=handle_request, args=(connected_socket,)).start()
    except socket.timeout:
        print("nothing new...")

I hope anyone can help me with this :D

Answer 1

I fixed the problem! (I think)

So, I followed what user253751 said about how HTTP is allowed to use the same connection for more than one request so I made adjustments and also so that that works it now has to detect the end-of-response/request (eor) for the responses and requests. First thing I did was wrap the whole handle_request code in a while statement for the multiple requests on one connection thing.

For the end-of-response I am using regex

eor = re.compile('\\r\\n\\r\\n\\Z')

and then at the appropriate place:

eorm = eor.search(str(response, 'ISO-8859-1', ignore)) # I replaced the utf-8
                                                       # by the correct ISO-
                                                       # 8859-1 used for HTTP 
                                                       # just to be safe

if eorm or len(response) == 0:
    safe_send(return_address, response)
    ipv4side.close()
    break

And for the end-of-request it's basically the same just it only looks that the request is 0 length. The code responsible for sending I put into the safe_send function that takes a connection and a msg.

Aaand I coded it so that in case the server aborts a connection for some reason and thus throws an error when trying to receive, it resends the request on a new connection.

try:
    response = ipv4side.recv(2048)
except ConnectionAbortedError:
    ipv4side = socket.create_connection(("127.0.0.1", 7245))
    safe_send(ipv4side, request)
    continue

I hope this is a good explanation :]