Python 多线程队列

Question

import threading
from queue import Queue



print_lock = threading.Lock()
def job(worker):
    with print_lock:
        with open('messages.txt') as f:
            for line in f:
                print(line)

def reader():
    while True:
        worker = q.get()
        job(worker)
        q.task_done()

q = Queue()

for x in range(10):
    t = threading.Thread(target=reader)

    t.daemon = True
    t.start()


for worker in range(1):
    q.put(worker)

q.join()

So what i want is each thread reads different messages,

Answer 1

Queue is thread safe

so, threadling lock does not needed

Answer 2

your trying too many things to learn in same code snippet like 1) Multi-Threading 2) Queue Data Structure 3) Thread Synchronization Mechanisms 4) Locking etc.

Let me answer regarding multi-threading only.

In your case, every thread is reading all messages because target function " job" is opening file and reading all the data and every thread is calling that target function.

Let me simplify the stuff bit.

1. You want to read each line of file in new thread. So, instead of opening file in every thread and read it, we will open file one time and put data in list.
2. Now, every thread will read one line from list and print it. Also, it will remove that printed line from list.
3. Once, all the data is printed and still thread trying to read, we will add the exception.

Code:

import threading
import sys


#Global variable list for reading file data
global file_data
file_data = []

#Create lock
lock = threading.Lock()


def reader():
    while len(file_data) != 0:
        print threading.currentThread().getName() + " --- "

        try:
            lock.acquire()
            #Get one line from list and print it
            a = file_data.pop()
            print a

        except:
            #Once data is not present, let's print exception message
            print "------------------------"
            print "No data present in file"
            sys.exit() 
        lock.release()

#Read data from file and put it into list
with open("messages.txt") as fh:
    file_data = fh.readlines()

for x in range(2):
    name = "Thread_"+str(x)
    t = threading.Thread(name=name,target=reader)
    t.start()

Output:

    C:\Users\dinesh\Desktop>python demo.py
Thread_0 --- Thread_1 ---
Each thread read each message

Thread_1 --- I am great


Thread_0 --- How Are you ?


Thread_1 --- Grey


Thread_0 --- Hey


Thread_1 --- Dinesh


Thread_0 --- Hello


------------------------
No data present in file

C:\Users\dinesh\Desktop>

    C:\Users\dinesh\Desktop>

NOTE: I know sue of global is not recommended. But for learning purpose it is good.