Multithreading - Alternating between two threads using Conditions and Events in Python

Question

I am trying to write a program using which I wish to alternate between two threads, thread1 and thread2. The tricky part is that the thread should begin execution first must be thread1. This is the code I have so far:

Class Client:
#member variables
def sendFile(self,cv1,lock1):

        sent=0;
        while (i<self.size):
            message = self.data[i:1024+i]
            cv1.acquire()
            BadNet.transmit(self.clientSocket,message,self.serverIP,self.serverPort)
            cv1.notify() 
            cv1.release()

            i = i+1024
            sent+=1
            lock1.wait()

        print "File sent successfully !"   
        self.clientSocket.close()

    def receiveAck(self,cv1,lock2):
        i=0
        while (1):
            lock1.clear()
            cv1.acquire()
            cv1.wait()
            print "\nentered ack !\n"
            self.ack, serverAddress = self.clientSocket.recvfrom(self.buf)

            cv1.release()
            lock1.set()


if __name__ == "__main__":
    lock1 = Event()
    cv1 = Condition()
    cv2= Condition()
    client = Client();
    client.readFile();

    thread1 = Thread(target = client.sendFile, args=[cv1,lock1])
    thread2 = Thread(target = client.receiveAck, args=[cv1,lock1])

    thread1.start()
    thread2.start()
    thread1.join()
    thread2.join()

The problem I am currently facing is that initially the program does alternate between two threads (confirmed by the output on the console. But after an arbitrary number of iterations (usually between 20 and 80) the program just hangs and no further iterations are performed.

Answer 1

There are at least two problems with your synchronization.

First, you're using cv1 wrong. Your receive thread has to loop around its cv, checking the condition and calling wait each time. Otherwise, you're just using a cv as a broken event + lock combination. You don't have such a loop. More importantly, you don't even have a condition to wait for.

Second, you're using lock1 wrong. Your receive thread sets the event and then immediately clears it. But there's no guarantee that the send thread has gotten to the wait yet. (The race from the previous problem makes this more of a problem, but it's still a problem even if you fix that.) On a multi-core machine, it will usually get there in time, but "usually" is even worse than never in threaded programming. So, eventually the send thread will get to the wait after the receive thread has already done the clear, and therefore it will wait forever. The receive thread, meanwhile, will be waiting to be notified by the send thread, which will never happen. So you're deadlocked.

For future reference, adding print statements before and after every blocking operation, especially sync operations, would make this a lot to debug: you would see the receive thread's last message was "receive waiting on cv1", while the send thread's last message was "send waiting on lock1", and it would be obvious where the deadlock was.

---

Anyway, I'm not sure what it would even mean to "fix" a cv with no condition, or an event that you're trying to use as a cv, so instead I'll show how to write something sensible with two cvs. In this case, we might as well just use a flag that we flip back and forth as the condition for both cvs.

While I'm at it, I'll fix a couple other problems that made your code not even testable (eg, i is never initialized), and include the debugging information, and what I had to fill in to make this a complete example, but otherwise I'll try to leave your structure and irrelevant problems (like Client being an old-style class) intact.

class Client:
    def __init__(self):
        self.clientSocket = socket(AF_INET, SOCK_DGRAM)
        self.serverIP = '127.0.0.1'
        self.serverPort = 11111
        self.buf = 4
        self.waitack = False

    def readFile(self):
        self.data = ', '.join(map(str, range(100000)))
        self.size = len(self.data)

    #member variables
    def sendFile(self,cv1,lock1):
        i = 0
        sent=0
        while (i<self.size):
            message = self.data[i:1024+i]
            print "s cv1 acquire"
            with cv1:
                print "s sendto"
                self.clientSocket.sendto(message, (self.serverIP, self.serverPort))
                self.waitack = True
                print "s cv1 notify"
                cv1.notify() 

            i = i+1024
            sent+=1

            print "s cv2 acquire"
            with cv2:
                print "s cv2 wait"
                while self.waitack:
                    cv2.wait()

        print "File sent successfully !"   
        self.clientSocket.close()

    def receiveAck(self,cv1,lock2):
        i=0
        while (1):
            print "r cv1 acquire"
            with cv1:
                while not self.waitack:
                    print "r cv1 wait"
                    cv1.wait()
            print "r recvfrom"
            self.ack, serverAddress = self.clientSocket.recvfrom(self.buf)
            i += 1
            print self.ack, i            

            print "r cv2 acquire"
            with cv2:
                self.waitack = False
                print "r cv2 notify"
                cv2.notify()

And here's a test server for it:

from itertools import *
from socket import *

s = socket(AF_INET, SOCK_DGRAM)
s.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
s.bind(('127.0.0.1', 11111))

for i in count():
    data, addr = s.recvfrom(1024)
    print(i)
    s.sendto('ack\n', addr)

Start the server, start the client, the server will count up to 672, the client will count up to 673 (since your code counts 1-based) with 673 balanced pairs of messages and a "File sent successfully !" at the end. (Of course the client will then hang forever because receiveAck has no way to finish, and the server because I wrote it as an infinite loop.)