Why does a pipe in write mode outputs something?

Question

While I was playing with pipes in c++ I stumbled accross something rather interesting.

#include <cstdio>
#include <iostream>
#include <string>

int main()
{
    FILE *pystream = popen("python","w"); // Calling the python console

    fprintf(pystream,"print(2+3)"); // Making it do something   

    pclose(pystream); // Closing the pipe

    return 0;
}

This code outputs 5. but why ? And can the "output" be read or stored somewhere ? I'm fairly new to C buffers and pipes, so I don't know if I'm using the right terminology.

Answer 1

When you write like this you're effectively writing to the stdin of the process you just started, in this case the python REPL. On Linux the python REPL is getting the expression directly ie it's not being typed in. This is ths system command

read(0, "print(2+3)", 4096) = 10

If you were doing this in the terminal each character is being read in one at a time by the terminal and when it gets carriage return it writes a newline \\n ie

read(0, "\r", 1)                        = 1
write(1, "\n", 1

It then performs the calculation and write the result out

write(1, "5\n", 25

You're by passing the terminal and writing the data directly to the stdin of the python interpreter. If you want to see how this can easily break try this code.

#include <cstdio>
#include <iostream>
#include <string>
int main()
{
    FILE *pystream = popen("python","w"); // Calling the python console
    fprintf(pystream,"print(2+3)"); // Making it do something   
    fprintf(pystream,"print(2+3)"); // Making it do something   
    pclose(pystream); // Closing the pipe
    return 0;
}

You will get a syntax error, to make it work the stdin needs to be fed a carriage return or a newline to separate the two lines ie add a carriage return...

fprintf(pystream,"print(2+3)\r");

Answer 2

The standard output of the command you're executing is connected to the standard output of your program, so when the Python writes to its standard output, it appears on the standard output of your process too.

If you had pending output before you ran Python, that won't be flushed and will appear after Python returns. For example,

std::cout << "Hello";

(no endl , no \\n in the string) before popen() and

std::cout << " World\n";

after pclose() means that you'll see the Python output before Hello World .

---

If you want to write to Python and read the results back in your program, you can no longer use popen() and pclose() . Instead, you need to use pipe() twice (one pipe to talk to Python, one pipe to read from Python), and you need to use fork() , exec() , dup2() — probably; dup() otherwise — and close() to make the operations work. You'll be using file descriptors and hence read() and write() system calls in the parent process, too.

Those are all C functions (system calls) more than C++ functions.

This code works:

#include <unistd.h>
#include <cstdio>
#include <cstring>

int main()
{
    int p1[2];
    int p2[2];
    if (pipe(p1) != 0 || pipe(p2) != 0)
        return 1;
    int pid;
    if ((pid = fork()) < 0)
        return 1;
    if (pid == 0)
    {
        dup2(p1[0], STDIN_FILENO);
        dup2(p2[1], STDOUT_FILENO);
        close(p1[0]);
        close(p1[1]);
        close(p2[0]);
        close(p2[1]);
        execlp("python", "python", (char *)0);
        fprintf(stderr, "failed to exec python\n");
        return 1;
    }
    else
    {
        close(p1[0]);
        close(p2[1]);
        const char command[] = "print(2+3)\n";
        int len = strlen(command);
        if (write(p1[1], command, len) != len)
        {
            fprintf(stderr, "failed to write command to python\n");
            return 1;
        }
        close(p1[1]);
        char buffer[256];
        int nbytes;
        if ((nbytes = read(p2[0], buffer, sizeof(buffer))) <= 0)
        {
            fprintf(stderr, "failed to read response from python\n");
            return 1;
        }
        printf("Python said: (%d) [%.*s]\n", nbytes, nbytes, buffer);
        close(p2[0]);
        printf("Finished\n");
    }
    return 0;
}

The bad news is that changing this code to write more than one command while synchronously reading a response from Python does not work. Python does not process each line separately as it does when its input is a terminal; it reads all the data before it responds at all. You can work around that with python -i , but then the prompts from Python appear on stderr . So, you can redirect that to /dev/null to lose it:

#include <unistd.h>
#include <fcntl.h>
#include <cstdio>
#include <cstring>

int main()
{
    int p1[2];
    int p2[2];
    if (pipe(p1) != 0 || pipe(p2) != 0)
        return 1;
    int pid;
    if ((pid = fork()) < 0)
        return 1;
    if (pid == 0)
    {
        dup2(p1[0], STDIN_FILENO);
        dup2(p2[1], STDOUT_FILENO);
        close(p1[0]);
        close(p1[1]);
        close(p2[0]);
        close(p2[1]);
        int dn = open("/dev/null", O_WRONLY);
        if (dn >= 0)
        {
            dup2(dn, STDERR_FILENO);
            close(dn);
        }
        execlp("python", "python", "-i", (char *)0);
        fprintf(stderr, "failed to exec python\n");
        return 1;
    }
    else
    {
        close(p1[0]);
        close(p2[1]);
        const char *commands[] =
        {
            "print(2+3)\n",
            "print(3+4)\n",
        };
        enum { NUM_COMMANDS = sizeof(commands) / sizeof(commands[0]) };
        for (int i = 0; i < NUM_COMMANDS; i++)
        {
            int len = strlen(commands[i]);
            if (write(p1[1], commands[i], len) != len)
            {
                fprintf(stderr, "failed to write command to python\n");
                return 1;
            }
            char buffer[256];
            int nbytes;
            if ((nbytes = read(p2[0], buffer, sizeof(buffer))) <= 0)
            {
                fprintf(stderr, "failed to read response from python\n");
                return 1;
            }
            printf("Python said: (%d) [%.*s]\n", nbytes, nbytes, buffer);
        }
        close(p1[1]);
        close(p2[0]);
        printf("Finished\n");
    }
    return 0;
}

Without redirection of stderr :

Python 2.7.10 (default, Oct 23 2015, 19:19:21) 
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> >>> Python said: (2) [5
]
>>> Python said: (2) [7
]
Finished

With redirection of stderr :

Python said: (2) [5
]
Python said: (2) [7
]
Finished

The disadvantage of losing the standard error output to /dev/null is that you won't get any notice when Python objects to what you send it to execute — the code will hang. Working around that is fun (a third pipe, and using poll() or epoll() or — perish the thought — select() would be one way around the problem).