Proxy server made with sockets (python) receiving and sending in incorrect order
Question
I was trying to make a simple proxy server, that can be used to relay any request message to some destination host, and receive the response back and relay it back to the original sender (client). My implementation causes the response of the first request message to get stalled in the server, and it only gets received as the response of the second request message. This causes some errors. I was trying to use this as an http proxy, running a simple http server on localhost, the proxy also on localhost, and used my browser to send GET messages to the proxy.
Below is the code
class Proxy:
def __init__(self,mhost,mport,dhost,dport):
self.mhost = mhost
self.mport = mport
self.dhost = dhost
self.dport = dport
#self.middle_man = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
#self.middle_man.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.colorAndFormat = ColorAndFormat() # Just for coloring the output text
def start_listener(self):
colorAndFormat = ColorAndFormat()
middle_man_recv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
middle_man_recv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
middle_man_recv.bind((self.mhost, self.mport))
middle_man_recv.listen()
print(colorAndFormat.Blue2("Listening on " + self.mhost + ':' + str(self.mport) + ' ...\n---------------------------------'))
middle_man_forw = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
middle_man_forw.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
#middle_man_forw.connect_ex((self.dhost, self.dport))
while True:
#try:
# Receive data
client_socket,client_addr = middle_man_recv.accept()
data = client_socket.recv(1024)
print(colorAndFormat.Blue2("Data received from client " + client_addr[0] + ':' + str(client_addr[1])))
print(colorAndFormat.Blue2(data.decode()))
# Forward data to dhost:dport
middle_man_forw.connect_ex((self.dhost, self.dport))
middle_man_forw.sendall(data)
print(colorAndFormat.Beige("Data forwarded to " + self.dhost + ':' + str(self.dport)))
# Receive data from dhost:dport
data = middle_man_forw.recv(1024)
print(colorAndFormat.Beige("Data received from " + self.dhost + ':' + str(self.dport)))
print(colorAndFormat.Beige(data.decode()))
# Forward data to client
client_socket.sendall(data)
print(colorAndFormat.Blue2("Data forwarded to client"))
print('---------------------------------')
client_socket.close()
#except
Below is the output on sending a GET request. (index.html exists)
Listening on 127.0.0.1:4444 ...
---------------------------------
Data received from client 127.0.0.1:60032
GET /index.html HTTP/1.1
Host: 127.0.0.1:4444
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Data forwarded to 127.0.0.1:8888
Data received from 127.0.0.1:8888
HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/3.8.3
Date: Tue, 17 Nov 2020 20:13:53 GMT
Content-type: text/html
Content-Length: 116
Last-Modified: Tue, 17 Nov 2020 19:49:32 GMT
Data forwarded to client
---------------------------------
Data received from client 127.0.0.1:60036
GET /favicon.ico HTTP/1.1
Host: 127.0.0.1:4444
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0
Accept: image/webp,*/*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Data forwarded to 127.0.0.1:8888
Data received from 127.0.0.1:8888
<html>
<head><title>This worksss!!</title></head>
<body>
If you're seeing this, the Proxy works!!
</body>
</html>
Data forwarded to client
---------------------------------
Data received from client 127.0.0.1:60038
GET /index.html HTTP/1.1
Host: 127.0.0.1:4444
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Traceback (most recent call last):
File "exposer.py", line 176, in <module>
exposer.start_listener()
File "exposer.py", line 40, in start_listener
middle_man_forw.sendall(data)
BrokenPipeError: [Errno 32] Broken pipe
Is this implementation wrong?
1 answers
solution1
0 2020-11-17 21:16:24
data = middle_man_forw.recv(1024)
You expect data to contain the full response of the server. This expectation is wrong. First, it might be more than the 1024 bytes you are trying to read. Then, recv might only return a part of the data, even if the total data are less than 1024 bytes.
The reason for this is that TCP is not a message protocol but just a byte stream. But you expect to recv full messages, with the assumption that a HTTP response is a single message.
There are basically two approaches on how to handle this: you can continue with your approach where you first expect the HTTP request, forward it, expect the HTTP response and forward it. But then you need to make sure that you actually read full requests and responses, ie you have to understand and properly implement the HTTP protocol.
The other approach would be to simply forward everything from client to server and in parallel forward everything from server to client. In this case you don't actually care what protocol is spoken but just forward everything blindly. This approach can be implemented with threads (one thread for each direction), select or similar. See for example A python proxy in less than 100 lines of code for the select-based approach.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.