error 10054 in a multithreaded server program in python

Question

I am trying to design a multithreaded server. client1 send data to server and server forwards data to client2 for processing. client2 sends processed data to server and then server forwards it to client1.

Data from client1 is received by client2 through server while execution, but my server program after sending data to client2 terminates with error

[10054] An existing connection was forcibly closed by remote host.

client1.py

from socket import *
import thread

ip='127.1.1.2'
port=5554

s=socket(AF_INET, SOCK_STREAM)
s.connect((ip,port))
data=raw_input("Input lowercase :: ")
while data!="@close":
    s.send(data)
    data=raw_input("Input next word or type @close to exit: ")

    s.close()

client2.py

from socket import *
import thread

ip='127.1.1.2'
port=5554

s=socket(AF_INET, SOCK_STREAM)
s.connect((ip,port))

m=s.recv(1024)
k=m.upper()
print "Uppercase of ",m," in System--> ",k
s.close()

server.py

from socket import *
import thread

ip='127.1.1.2'
port=5554
data=''
def multi_threading(c,a):
    while True:
        global data
        print "Inside client 1 thread"
        data= c.recv(1024)
        if not data:
            break
        if "close"==data.rstrip():
            break
    c.close()
    print a,"--terminates connection"

def multi_threading2(c,a):
    while True:
        print "Inside client 2 thread"
        c.send(data)
        if not data:
            break
        if "close"==data.rstrip():
            break
    c.close()
    print a,"--terminates connection"



if __name__=='__main__':

        s=socket(AF_INET, SOCK_STREAM)
        s.setsockopt(SOL_SOCKET, SO_REUSEADDR,1)
        s.bind((ip,port))
        s.listen(4)
        count=0
        while True:
            print "Server is listening on port number ",port
            c,a=s.accept()
            count=count+1
            print "--------------------------------------------------"
            print "Server connected to system having ip address --> ",a
            if(count==1):
                thread.start_new_thread(multi_threading,(c,a))
            else:
                thread.start_new_thread(multi_threading2,(c,a))

What i might be doing wrong?

Answer 1

I'm trying to help by changing your code as little as possible. It isn't how I would write it but should allow you to follow along. This is also not very idomatic Python. I just want you to see how to get the basic interaction going.

Some comments about my changes:

1. Remove the indent in client1.py so you don't always close the socket with s.close(). I suspect this was a typo in your post?
2. In server.py it isn't safe to access the global data between two threads. At the very least you need a semaphore to protect it.
3. Every time around multi_threading loop you destroy the last value of data. You need to append the incoming bytes to a data structure like a list. I renamed the threads to make it easier to understand.
4. You are expecting to see close in the server but you don't ever send this because your loop terminates when you type @close . You need to adjust for this.
5. Your code is order dependent because both client 'sender' and client 'receiver' use the same port. You need to put something into the protocol to distinguish between these two roles so you can start them in either order. This makes the system more robust.
6. Due to the sender not necessarily having any data I do a crude sleep. This can be avoided by using a shared queue between the threads which would block one end
7. You need some framing because you can't guarantee the speed at which the independent processes send and receive to the socket. I use \\n for this.
8. There is an edge case if you sent very large strings (and removed the 1024 read) you could not get a full word with the \\n through. I ignore this case.
9. I adjust the loopback address

I wrote this in PyCharm CE. I recommend using an IDE (PyCharm is excellent) with breakpoints. You can step through all the processes bit by bit. Put breakpoints in the threads and step through the actions.

Result:

server.py

from socket import *
import thread
from threading import Lock
import time

ip = '127.0.0.1'
port = 5554
data = []
data_lock = Lock()


def multi_threading_recv(sock, a):
    should_run = True
    while should_run:
        global data
        print "Inside client 1 thread"
        _data = sock.recv(1024)
        if _data:
            with data_lock:
                for word in _data.split('\n'):
                    data.append(word)
                    if "@close" == word:
                        should_run = False
                        break
        else:
            break

    sock.close()
    print a, "--terminates connection"


def multi_threading_send(sock, a):
    while True:
        print "Inside client 2 thread"
        with data_lock:
            if len(data) == 0:
                time.sleep(1)  # Not ideal.  Ideally block on a shared queue
                continue
            else:
                _data = data.pop(0)
        if _data == "@close":
            break
        sock.send(_data + '\n')
    c.close()
    print a, "--terminates connection"

if __name__ == '__main__':
        s = socket(AF_INET, SOCK_STREAM)
        s.setsockopt(SOL_SOCKET, SO_REUSEADDR,1)
        s.bind((ip,port))
        s.listen(4)

        count = 0
        while True:
            print "Server is listening on port number ", port
            c, a = s.accept()
            print "--------------------------------------------------"
            print "Server connected to system having ip address --> ", a
            role = c.recv(4)
            if role == 'PUSH':
                thread.start_new_thread(multi_threading_recv, (c, a))
            elif role == 'PULL':
                thread.start_new_thread(multi_threading_send, (c,a))
            else:
                print('invalid role: ' + role)
                c.close()

client1.py

from socket import *

ip = '127.0.0.1'
port = 5554

s = socket(AF_INET, SOCK_STREAM)
s.connect((ip,port))
s.send('PUSH')
while True:
    data = raw_input("Input word or type @close to exit: ")
    s.send(data + '\n')
    if data == "@close":
        break
s.close()

client2.py

from socket import *

ip = '127.0.0.1'
port = 5554

s = socket(AF_INET, SOCK_STREAM)
s.connect((ip, port))
s.send('PULL')

while True:
    data = s.recv(1024)
    if data:
        words = data.split('\n')
        for w in words:
            print(w)
    else:
        break
s.close()

Output:

client1

Input word or type @close to exit: hello
Input word or type @close to exit: world
Input word or type @close to exit: @close

client2:

hello
world

server:

Server is listening on port number  5554
--------------------------------------------------
Server connected to system having ip address -->  ('127.0.0.1', 62605)
Server is listening on port number  5554
Inside client 1 thread
('127.0.0.1', 62605) --terminates connection
--------------------------------------------------
Server connected to system having ip address -->  ('127.0.0.1', 62614)
Server is listening on port number  5554
Inside client 2 thread
Inside client 2 thread
Inside client 2 thread
('127.0.0.1', 62614) --terminates connection

There is one final comment that the server will go around the loop again and keep on accepting clients. The code doesn't cope with this and I leave it as an exercise for you. You can multiplex multiple senders and receivers if you add a channel number to the PUSH and PULL initiation of the protocol (eg PUSH3, PULL3). This will then allow you to store the incoming data in a dictionary keyed by this integer and then send it out to the correct client.