PyQT Threading and Socketing safely, and catching multiple signals

Question

I'm writing some code currently that uses standard threads, a Queue, QThreads in pyqt, and sockets. Basically I open a socket server, and receive data from other programs. I have a few questions here.

1. My program seems to hang upon closing even if I try to kill my threads properly. The only way to avoid this is to set a particular thread as a daemon. What I was doing to kill it, was using a member( self._terminate = threading.Event() ) which I would set in my closeEvent() of my GUI. My thread would check constantly for this termination and return if it exists. My socket would also be closed - child connections would have shutdown() called on them, and the main server would have some fake data sent through to push the listen loop on the thread, which would also terminate.

All of this, as I mentioned, happened in the closeEvent() of my GUI, but when I had it set up this way, it would hang on exit. The program would hide but the shell would remain active, and it wasn't dead. By setting the thread that holds the server socket to a daemon, everything closed okay.

This is safe to do, since I am not writing any contents to a file, or reading from a file, correct?

2. My second part of the question, is I have a QThread set up to collect any incoming data form my PriorityQueue and send signals to the GUI as needed. It sends all available data each time it does a check, something roughly like:

-

class DataCollector(QtCore.QThread):
    dataSig = QtCore.SIGNAL('data')

    def __init__(self, q):
        super(DataCollector, self).__init__(None)

        self.check_time = 5
        self.scan_time = 0.2
        self.last_check = time.time() - self.check_time
        self.queue = q
        self._terminate = False

    def run(self):
        while True:
            if self._terminate:
                return

            time.sleep(self.scan_time)

            if time.time() > self.last_check + self.check_time:
                while self.queue.empty is False:
                    data = self.queue.get()
                    self.emit(self.dataSig, data)            

My main UI would then catch these signals, and in my tests - add them to a last widget.

In my tests, once in a while, the test data would get combined into one line - for instance if my Queue contained

• test
• test again
• test MORE

and i emitted these signals quickly until the queue was empty, once in a while my list widget would have a double entry such as:

• test
• test again
• test MORE
• test
• test again
• test MORE
• test
• test againtest More ******

Is this because I'm firing them out too fast? What might cause that, and what are some possible fixes?

Thanks for any and all input!

Answer 1

I suspect it has nothing to do with the speed of the signal, since QT internally handles these in a queue-type fashion. What I think is happening, is that you are under the impression that when you call send or sendall from a socket, all the data is sent as one packet, and when you recv(bytes) , it will receive the full packets sent one at a time.

This is not the case. The data is sent in a stream like fashion, so you can end up with chunks, or combined results. There are 2 ways around this:

1. Send some data at the start of each packet saying how long, in bytes, the following data is, and then recv that much.

2. Send your data with a delimiter, for example, if your delimiter is |!| , the data sent would be: test|!| , testAgain|!| , testMore|!| . Then you handle this on your receiving thread. For instance, the stream may come through the first time as: test|!|testA , followed by gain|!|testMore|!| the second time.

In your thread you can handle splitting the received data by the given delimiter, and holding onto unfinished chunks until the next delimiter is found.