Multi-threaded multi-client server in python

Question

I'm writing a multi-threaded, multi-client server in python. Multiple users can connect to it with telnet and basically use it as a chat server. I'm able to connect with two clients through telnet, but I run into the two following problems:

1. The first client to send a message is immediately disconnected.
2. The other client does not the receive the message sent by the first client.

Server code:

import os
import sys
import socket
import thread

port = 1941
global message
global lock
global file

def handler(connection):
    while 1:
            file = connection.makefile()
            file.flush()
            temp = file.readline()
            if temp == 'quit':
                break
            lock.acquire()
            message += temp
            lock.release()
            file.write(message)
    file.close()

acceptor = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
acceptor.bind(('', port))
acceptor.listen(10)
lock = thread.allocate_lock()

while 1:
    connection, addr = acceptor.accept()
    thread.start_new_thread(handler, (connection,))

Ok I listened to unholysampler and now I have this. I'm able to to connect with both clients now and type messages, but they aren't being sent/received (I can't tell which one).

import os
import sys
import socket
import thread

port = 1953

def handler(connection):
    global message
    global filelist
    filelist = []
    file = connection.makefile()
    file.flush()
    filelist.append(file)
    message = ''
    while 1:
        i = 0
        while i < (len(filelist)):
            filelist[i].flush()
            temp = filelist[i].readline()

            if temp == 'quit':
                break

            with lock:
                message += temp

            i = i + 1
    file.close()

global lock
acceptor = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
acceptor.bind(('', port))
acceptor.listen(10)
lock = thread.allocate_lock()

while 1:
    connection, addr = acceptor.accept()
    thread.start_new_thread(handler, (connection,))

Answer 1

It's much simpler and better to implement this sort of thing using Twisted , which lets you handle multiple clients concurrently in a single thread, as well as providing a nicer API.

Here's how you write a chat server using Twisted (full example in chatserver.py ):

class MyChat(basic.LineReceiver):
    def connectionMade(self):
        print "Got new client!"
        self.factory.clients.append(self)

    def connectionLost(self, reason):
        print "Lost a client!"
        self.factory.clients.remove(self)

    def lineReceived(self, line):
        print "received", repr(line)
        for c in self.factory.clients:
            c.message(line)

    def message(self, message):
        self.transport.write(message + '\n')

For each user, a MyChat object gets created, and the event loop calls its methods for start/stop events and when a line is received from the client. In this case, it just send every line it receives to all the clients in the system. Since it runs in a single thread, no locks are needed.

Answer 2

That's not how you use global . When you define a method, inside the method scope, you use the global command to make references to the variable the the higher scoped variable.

message = 1
def globalTest():
  global message
  message += 1
  print message

print message
globalTest()
print message

You are making a new file object for the connection every time you iterate over the loop. You want to make that before the loop starts so you are only doing it once.

You are reading from and writing to the same file object. This means it is just an echo server. You are never giving thread1 a reference to thread2's file. Trying to use one global variable for the socket file will not work because you will never know which socket it is actually pointing to. (Issue #2)

You never initialize message, so message += temp will throw an UnboudLocalError saying that it is being referenced before assigned a value. (Likely the cause of issue #1) Also, why are you appending the string in the first place, that means every time something is sent, the entire conversation is sent out.

Also, don't manually acquire and release the lock, using with is cleaner.

with lock:
  message += temp

Answer 3

I think you need to call s.listen before every single connection. That is put it inside the infinite loop. while True: acceptor.listen(1) #...