python multithreading using python

Question

I was going through a simple multithreading tutorial . In which there were 10 threads created corresponding to handle 20 workers . I got the point how a thread handles a worker , and 10 threads runs in parallel serving 20 workers which are residing in a priority queue. But my question is , what will happen if one thread has completed serving the assigned worker , and 9 threads are still serving their workers.

I want to create a script in which if a thread's work is over then it should fetch the others worker residing in the queue.

here is my code :-

import threading
from queue import PriorityQueue
import time

print_lock = threading.Lock()

q = PriorityQueue()

def threader():
    i = 0
    while True:
        worker = q.get()
        exampleJob(worker)
        q.task_done()



for x in range(10):
    t = threading.Thread(target = threader)
    t.deamon = True
    t.start()

start_time = time.time()

for worker in range(1,20):
    q.put(worker)

def exampleJob(worker):
    if(worker ==1):
        time.sleep(0.5)
    else:
        time.sleep(5)

    with print_lock:
        print(threading.current_thread().name,worker)

q.join()


print('Entire job took:', time.time()-start_time)

In above code I have created a sleep of 0.5 for worker 1 but rest have 5 sec sleep , if I execute this script, Thread 1 handles worker 1 and waits until all other 9 threads have completed their worker . I want if thread 1 have completed the work it should take worker number 11 to process.

output of above code is:-

Thread-1 1
Thread-3 3
Thread-5 5
Thread-4 4
Thread-2 2
Thread-6 6
Thread-10 10
Thread-7 7
Thread-9 9
Thread-8 8
Thread-1 11
Thread-3 12
Thread-5 13
Thread-4 14
Thread-2 15
Thread-6 16
Thread-10 17
Thread-7 18
Thread-9 19
Entire job took: 10.137013912200928

Answer 1

You created 10 threads and started each of them ( for x in range(10): ).

Each of the threads will try to retrieve an object from the queue. Since the queue's get method blocks, the first thread that actually gets to call .get() first will block until something is placed in the queue. The next thread will block until the .get() returns in the previous thread. This is guaranteed only because the documentation for that Queue class says that it is thread-safe. If a class or data structure is not documented (and actually implemented correctly) as being thread-safe then all bets are off.

Note that threading gives you no guarantee regarding order unless you yourself explicitly code the necessary synchronization (such as using locks or similar). So there is no guarantee that Thread 1 will be the first to actually begin executing and enter the .get() method.

After you have started each of the 10 threads you then begin putting numbers into the queue. As the numbers arrive in the queue, at some point the threads will wake up and receive them. The order of the threads receiving from the queue is not guaranteed, although the order of the values in the PriorityQueue is guaranteed.

When it is scheduled to run, each thread will sleep for the amount of time you specified. Note that sleep() provides very little guarantee as to the actual amount of time that will elapse between calling the function and returning from the function.

The actual suspension time may be less than that requested because any caught signal will terminate the sleep() following execution of that signal's catching routine. Also, the suspension time may be longer than requested by an arbitrary amount because of the scheduling of other activity in the system.

Changed in version 3.5: The function now sleeps at least secs even if the sleep is interrupted by a signal, except if the signal handler raises an exception (see PEP 475 for the rationale).

So each thread will start it's sleep around the same time, but not exactly, and will wake up around the same time, since the sleep duration was the same, and be scheduled to execute around the same time. Note that threads can be suspended at any point during their execution and another thread can resume. On a single-core system only one thread can actually be executing at any given time and all other threads are waiting for the scheduler to suspend the one that is executing and switch to a different thread. On a multi-core system it is possible for separate threads to each run at the same time on separate cores, but each core will run at most one thread at a time.

After your threads return from .sleep() , they then contend to acquire the lock. This causes each one to finish printing its output before another starts printing. This is a good thing so that the output does not interleave in the middle. The order in which the threads acquire the lock is not guaranteed. The guarantee is that only one thread will have the lock at a time. If the lock is available then a thread will acquire it with no delay, but if it is not then it will wait until it is.

In the output you provided, Thread-1 did process "worker" 11. This isn't guaranteed, though, unless you ensure that you do not start any other thread until Thread-1 has called .get() on the queue. It is fairly predictable, though not guaranteed, in your program that whatever thread processes "worker" 1 will also process "worker" 11 because the sleep duration is so much shorter that it is very likely it will be next on the queue while the other nine threads are still in their call to .sleep() . It isn't guaranteed because the scheduler may not actually schedule that thread to execute; the thread may remain suspended and a different thread could reach the point of retrieving the next item from the queue before this one does.

Hopefully this explanation of your provided program is helpful.