Linux. Python. Read from named pipe

Question

I am having issues replicating some functionality in python.
Working under ubuntu.
I have a program in c++ that does stuff, and every once in a while it writes formatted data to a named pipe. I also have a c++ program that reads from that named pipe. No issues there. However I have almost no clue as to how to replicate the same thing (only the reading part) in python.

Here's the c++ code I use to read:

//various includes
#define FIFO_NAME "/tmp/MYFIFO"

using namespace std;

int main()
{
    int fp;
    fp = open(FIFO_NAME, O_RDONLY);
    char readbuf[80];
    while (true)
    {
        int tot = 0;
        //read an int to know how many results are there to read right now
        while(tot<sizeof(int))
            tot+= read(fp, readbuf+tot, sizeof(int)-tot);
        int nres = *(int *)&readbuf[0];
        if (nres>0)
        {
            cout << nres << " results:" << endl;
            for (int i = 0; i < nres; i++)
            {
                //here I read 3 floats
            }
        }
    }
}

This c++ code works exactly as I want it to: it reads the bytes from the named pipe, converts them to the corresponding numbers, makes them available for me to play with.
I tried to do the same in python, but failing.
First attempt at python code:

import os
import time

FIFO_NAME = '/tmp/MYFIFO'

def main():
    fp = os.open(FIFO_NAME, os.O_RDONLY)
    while True:
        time.sleep(0.001)
#Here I just try to read the first 4 bytes (the initial int) and fail already
        line = os.read(fp, 4) #I think this would read 4 bytes from the FIFO?
        print(len(line)) #ALWAYS prints 4
        print(line) #ALWAYS prints a blank line
        print(int(line)) #this of course gives an error, saying that line ==''

Second attempt at python code (gives the same wrong results):

import os
import time

FIFO_NAME = '/tmp/MYFIFO'

def main():
    fp = open(FIFO_NAME, 'r')
    while True:
        time.sleep(0.001)
#Here I try to read up to the line break
        line = fp.readline() #I think this would read to the line break, but it actually returns immediately
        print(len(line)) #after the first few iterations, this always prints 6
        print(line) #ALWAYS prints a blank line

In the second attempt I was expecting to read up to the first linebreak (i can easily have the c++ producer put a linebreak at the end of each "packet"). However this always reads empty lines.

My question is about getting in python the functionalities I have in c++. I can see this happening in two ways:

• Fix the way I use the pipe on the python side, by actually reading the bytes as I do in c++ and marshaling those to the correct data types and values.
• Have the python "load" a buffer of bytes until it receives the linebreak, then extract my numbers from there.

Either way, I don't understand why the read and the readline are returning those blank strings, nor I know how to handle them.

I hope some of you guys can help me. Feel free to ask for clarifications and more information.

Answer 1

In my comment, I was referring to python, but that is by the by.
Take a gander at this, it may help you, as it is specifically about "Named Pipes between C# and Python".
http://jonathonreinhart.blogspot.co.uk/2012/12/named-pipes-between-c-and-python.html

In case the link goes missing, here is what jonathon says:

C#

// Open the named pipe.
var server = new NamedPipeServerStream("NPtest");

Console.WriteLine("Waiting for connection...");
server.WaitForConnection();

Console.WriteLine("Connected.");
var br = new BinaryReader(server);
var bw = new BinaryWriter(server);

while (true) {
    try {
        var len = (int) br.ReadUInt32();            // Read string length
        var str = new string(br.ReadChars(len));    // Read string

        Console.WriteLine("Read: \"{0}\"", str);

        str = new string(str.Reverse().ToArray());  // Just for fun

        var buf = Encoding.ASCII.GetBytes(str);     // Get ASCII byte array     
        bw.Write((uint) buf.Length);                // Write string length
        bw.Write(buf);                              // Write string
        Console.WriteLine("Wrote: \"{0}\"", str);
    }
    catch (EndOfStreamException) {
        break;                    // When client disconnects
    }
}

Console.WriteLine("Client disconnected.");
server.Close();
server.Dispose();

Python

import time
import struct

f = open(r'\\.\pipe\NPtest', 'r+b', 0)
i = 1

while True:
    s = 'Message[{0}]'.format(i)
    i += 1

    f.write(struct.pack('I', len(s)) + s)   # Write str length and str
    f.seek(0)                               # EDIT: This is also necessary
    print 'Wrote:', s

    n = struct.unpack('I', f.read(4))[0]    # Read str length
    s = f.read(n)                           # Read str
    f.seek(0)                               # Important!!!
    print 'Read:', s

    time.sleep(2)

In this example, I implement a very simple protocol, where every "message" is a 4-byte integer (UInt32 in C#, 'I' (un)pack format in Python), which indicates the length of the string that follows. The string is ASCII. Important things to note here:

Python The third parameter to open() means "unbuffered".
Otherwise, it will default to line-buffered, which means it will wait for a newline character before actually sending it through the pipe.

I'm not sure why, but omitting the seek(0) will cause an IOError #0.
I was clued to this by a StackOverflow question.