使用线程模块在python中实现线程

Question

In a python program I need 2 threads based on Threading module. Thread#1 generates some data and put it in a buffer and thread#2 is supposed to process the data in the buffer.

So my pseudocode is like this: Thread 1:

Thread#1
while True:
   Generate_Some_Data()
   while flag==1:
      pass()
   Buffer_Address=Write_It_To_Buffer()
   flag=1


Thread#2
while True:
   while flag==0:
      pass()
   Process_Data(Buffer_Address)
   flag=0

(Let us assume that access to the variable "flag" is atomized with suitable locks.)

I know that Threading module is not concurrent. Essentially, this means that unless one of the threads does not block on some external condition (such as file-io or time.sleep), both threads will share the total process time regardless of the tasks. Therefore, according to my understanding, approximately half of the total processing time will be wasted on "while flag" loops in the above configuration.

So, here are my questions:

1. Am I right in my above anticipation/understanding of the Threading module? Is the half of the total process time wasted in "while flag" loops?

2. (If I am right) Is there anyway to get rid of "while flag" loops completely? I tried to find out another structure in which I can make threads sleep in lock.acquire() methods, however, I could not figure out a %100 safe way of doing so. (When one thread relases lock, there is no quarantee that the other thread will acquire it before the same thread acquire it again)

Answer 1

It sounds like the Queue module is what you need.

This will give you a blocking FIFO queue. Whenever the consumer thread gets something from the queue, if there's nothing in there, it will block until something becomes available, giving the producer thread time to produce something.

If you're concerned about the producer thread hogging all the compute cycles, you can put a limit on the size of the queue. If the queue is full, and the producer thread has something to add, it will block until the consumer thread takes something out.

Here's some pseudocode:

import Queue
q = Queue.Queue()

# Producer thread
while True:
    data = generate_some_data()
    q.put(data)

# Consumer thread
while True:
    data = q.get()
    process_data(data)