Python script with multiple threads works normally only in debug mode

Question

I am currently working with one Python 2.7 script with multiple threads. One of the threads is listening for JSON data in long polling mode and parse it after receiving or go into timeout after some period. I noticed that it works as expected only in debug mode (I use Wing IDE). In case of just normal run it seems like this particular thread of the script hanging after first GET request, before entering the "for" loop. Loop condition doesn't affect the result. At the same time other threads continue to work normally.

I believe this is related to multi-threading. How to properly troubleshoot and fix this issue?

Below I put code of the class responsible for long polling job.

class Listener(threading.Thread):

def __init__(self, router, *args, **kwargs):
    self.stop = False

    self._cid = kwargs.pop("cid", None)
    self._auth = kwargs.pop("auth", None)
    self._router = router
    self._c = webclient.AAHWebClient()

    threading.Thread.__init__(self, *args, **kwargs)

def run(self):
    while True:
        try:
            # Data items that should be routed to the device is retrieved by doing a 
            # long polling GET request on the "/tunnel" resource. This will block until
            # there are data items available, or the request times out
            log.info("LISTENER: Waiting for data...")

            response = self._c.send_request("GET", self._cid, auth=self._auth)

            # A timed out request will not contain any data             
            if len(response) == 0:
                log.info("LISTENER: No data this time")             
            else:
                items = response["resources"]["tunnel"]
                undeliverable = []

                #print items # - reaching this point, able to return output

                for item in items:

                    # The data items contains the data as a base64 encoded string and the 
                    # external reference ID for the device that should receive it
                    extId = item["extId"]
                    data = base64.b64decode(item["data"])

                    # Try to deliver the data to the device identified by "extId"
                    if not self._router.route(extId, data):
                        item["message"] = "Could not be routed"
                        undeliverable.append(item)

                # Data items that for some reason could not be delivered to the device should
                # be POST:ed back to the "/tunnel" resource as "undeliverable"
                if len(undeliverable) > 0:
                    log.warning("LISTENER: Sending error report...")
                    response = self._c.send_request("POST", "/tunnel", body={"undeliverable": undeliverable}, auth=self._auth)

            except webclient.RequestError as e:
                log.error("LISTENER: ERROR %d - %s", e.status, e.response)

UPD:

class Router:
def route(self, extId, data):

    log.info("ROUTER: Received data for %s: %s", extId, repr(data))
    # nothing special
    return True

Answer 1

If you're using the CPython interpreter you're not actually system threading :

CPython implementation detail: In CPython, due to the Global Interpreter Lock, only one thread can execute Python code at once (even though certain performance-oriented libraries might overcome this limitation). If you want your application to make better use of the computational resources of multi-core machines, you are advised to use multiprocessing. However, threading is still an appropriate model if you want to run multiple I/O-bound tasks simultaneously.

So your process is probably locking while listening on the first request because your are long polling.

Multi-processing might be a better choice. I haven't tried it with long polling but the Twisted framework might also work in your situation.