Why is output being written to a pipe being buffered

Question

I am writing a program where the parent process needs to be able to communicate with a another child process, so i wrote a function that redirects the child's standard output and input to pipes.

int create_child_piped_io(int* pipe_in, int* pipe_out, const char* program, char* argv[], char* envp[])
{
    int stdin_pipe[2];
    int stdout_pipe[2];
    int child;


    if(pipe(stdin_pipe) < 0)
    {
        return -1;
    }

    if(pipe(stdout_pipe) < 0)
    {
        close(stdin_pipe[0]);
        close(stdin_pipe[1]);
    }
    
    child = fork();
    

    if(child == 0)
    {
        close(stdin_pipe[1]);
        close(stdout_pipe[0]);

        if(dup2(stdin_pipe[0], STDIN_FILENO) < 0)
        {
            close(stdin_pipe[0]);
            close(stdout_pipe[1]);

            exit(errno);
        }

        if(dup2(stdout_pipe[1], STDOUT_FILENO) < 0)
        {
            close(stdin_pipe[0]);
            close(stdout_pipe[1]);

            exit(errno);
        }

        close(stdin_pipe[0]);
        close(stdout_pipe[1]);

        execve(program, argv, envp);

        exit(1);
    }
    else if(child > 0)
    {
        close(stdin_pipe[0]);
        close(stdout_pipe[1]);

        *pipe_in = stdin_pipe[1];
        *pipe_out = stdout_pipe[0];

        return child;
    }
    else
    {
        close(stdin_pipe[0]);
        close(stdin_pipe[1]);
        close(stdout_pipe[0]);
        close(stdout_pipe[1]);


        exit(errno);
    }

    return child;

}


int main()
{
    int input, output;
    char* argv[] = {"program"};
    char* envp[] = {NULL};
    int cpid = create_child_piped_io(&input, &output, "/home/jaden/dev/c++/pipes/program", argv, envp);


    
    char c;


    while(read(output, &c, 1) == 1)
    {
        printf("%c", c);
    }
    printf("done\n");
    int status;
    waitpid(cpid, &status, 0);
    
    close(input);
    close(output);

}

this worked fine but i noticed when writing to the stdout from the child process it wouldn't get sent to the pipe immediately. this is the program the child process would run.

int main()
{
    // i < 1025 gets sent to pipe after sleep(10)
    // i > 1025 send first 1024 bytes to pipe immediately
    for(int i = 0; i < 1024; i++)
    {
        write(1, "a", 1);
    }
    
    sleep(10);
    return 0;
} 

the output would only get sent to the pipe after the child process ended. I tried different amounts of data to send. it turns out every time i write more than 1024 to stdout, it gets sent to the pipe. this leads me to believe that it is being buffered. I don't understand what the why this would be as the pipe is a buffer in memory as well, so it's just buffering the buffer. if this is standard behavior is there any way to turn it off, as i would like to get the data as soon as it is written.

i am using linux mint 20.3.

Answer 1

Your problem is here:

    while(read(output, &c, 1) == 1)
    {
        printf("%c", c);
    }

While you are reading with a syscall, you are writing through glibc stdio using printf() connected to a terminal. The output is Line-Buffered . If you change your code to write with a syscall as well, eg

    while(read(output, &c, 1) == 1)
    {
        write (1, &c, 1);
    }

The mysterious buffering disappears.

As @ShadowRanger points out, if you need the formatting provided by the stdio.h functions, then you can use setvbuf() to change the normal line-buffering of stdout when connected to a terminal to no buffering with. eg setvbuf(stdout, NULL, _IONBF, 0); This will enable output through printf() , etc.. to be used in an unbuffered manner. See man 3 setbuf