Different behavior with pipes between python threads and processes?

Question

I've got a python script that writes some data to a pipe when called:

def send_to_pipe(s):
    send = '/var/tmp/mypipe.pipe'
    sp = open(send, 'w')
    sp.write(json.dumps(s))
    sp.close()

if __name__ == "__main__":
    name = sys.argv[1]
    command = sys.argv[2]
    s = {"name":name, "command":command}
    send_to_pipe(s)

Then I have this file that keeps the pipe open indefinitely and reads data in every time the above script is called:

def watch_pipe():
    receive = '/var/tmp/mypipe.pipe'
    os.mkfifo(receive)
    rp = os.open(receive, os.O_RDWR | os.O_NONBLOCK)
    p = select.poll()
    p.register(rp, select.POLLIN)
    while True:
        try:
            if p.poll()[0][1] == select.POLLIN:
                data = os.read(rp,512)
                # Do some stuff with the data
        except:
            os.close(rp)
            os.unlink(receive)

if __name__ == "__main__":
        t = Thread(target=watch_pipe)
        t.start()
        # Do some other stuff that goes on indefinitely

This code works perfectly when I use threads. The pipe stays open, the first file writes to the pipe, and the stuff gets done. The problem is I can't stop the thread when I want to close the program. So I switched from Thread to Process:

        p = Process(target=watch_pipe)
        p.start()

But with a process instead of a thread, when I run the writer script, open(send, 'w') deletes the pipe as if it were a file I wanted to overwrite. Why is this? The permissions and ownership of the file is the same in both cases, and the writer script does not change. The only thing that changed was replacing a Thread object with an analogous Process object.

EDIT: After changing the open to use 'a' instead of 'w', the pipe still disappears when using a process.

Answer 1

Just an idea, use another opening mode for your file such as "a" instead of "w" . Indeed where "w" truncates the file, "a" only appends to the file.

I can imagine that when you use threads, ie in a single same process, the pipe file is already opened for reading when you open it for writing. In such a case, opening for writing maybe don't truncate the file. On the opposite with processes, the writing process does not know that the file is opened for reading and may delete it to truncate it.

Answer 2

Why are you making it difficult on yourself? You opened the FIFO as O_RDWR in the routine you said you were using in a thread. If you want to shut down the thread just have whatever other thread is shutting it down write a "stop" msg to the FIFO. The thread reads the msg, shuts itself down, and all is right again with the world. No fuss, no muss, no need for a separate process.