如何使用Python中的线程模块写入命名管道？

Question

I am trying to write a program which communicates to an external program through named pipes. The python script does not know when the external program opens up the named pipe/file for reading, so the python script need to open the pipe in blocking mode, see open . If the python script opens in non-blocking mode and the external program has not opened for reading, the open statement will result in an error.

So to open in blocking mode, the python script could open the named pipe in a separate thread, and I have tried the threading module. In the below example I just read from the named pipe in the main thread, but it produces the same error:

import threading
import os

pipe_name = 'pipe_test'


class WriterNamedPipe(threading.Thread):

    def __init__(self, filepath, input):
        '''
        Write: generate that will output each line of input
        '''
        # Inherit
        threading.Thread.__init__(self, verbose = True)
        self.daemon = False
        self.filepath = filepath
        self.input = input
        self.start()

    def run(self):
        # Open blockingly
        with open(self.filepath, 'w') as f:
            f.write(self.input)

if not os.path.exists(pipe_name):
    os.mkfifo(pipe_name)

WriterNamedPipe(pipe_name, '1\n' * 100)

with open(pipe_name, 'r') as f:
    print f.read()

This would results in a hang/freeze:

MainThread: <WriterNamedPipe(Thread-1, initial)>.start(): starting thread
Thread-1: <WriterNamedPipe(Thread-1, started 1078922160)>.__bootstrap(): thread started
Thread-1: <WriterNamedPipe(Thread-1, started 1078922160)>.__bootstrap(): normal return

Compilation hangup

However, a similar example from here works, but with os.fork :

import os, time, sys
pipe_name = 'pipe_test'

def child( ):
    pipeout = os.open(pipe_name, os.O_WRONLY)
    counter = 0
    while True:
        time.sleep(1)
        os.write(pipeout, 'Number %03d\n' % counter)
        counter = (counter+1) % 5

def parent( ):
    pipein = open(pipe_name, 'r')
    while True:
        line = pipein.readline()[:-1]
        print 'Parent %d got "%s" at %s' % (os.getpid(), line, time.time( ))

if not os.path.exists(pipe_name):
    os.mkfifo(pipe_name)  
pid = os.fork()    
if pid != 0:
    parent()
else:       
    child()

Why is the example with the threading module hanging?

Answer 1

This does probably not work due to the GIL . The open statement in the thread blocks the whole program. This could be avoided by using multiprocessing module instead.

Answer 2

sure as @hakanc pointed out due to GIL it will block your process. The os.fork is working because it actually spawn an subprocess instead of thread, which is not blocked by GIL.