ls | wc using C doesn't work

Question

I'have written a C program which uses multiple piping to simulate shell. The problem is I can run most of the commands like ls | cat ls | cat etc, but I'm unable to use ls | wc ls | wc . Is there any case when wc doesn't work?

int pipefd[4]; 
int p1 = pipe(pipefd);          // Open pipe 1
int p2 = pipe(pipefd + 2);      // Open pipe 2

pid_t pid;

for(i = 0; i < n_commands; i++)
{
    fflush(stdout);
    pid = fork();

    if(pid == 0)
    {
        int command_no = i;
        int prev_pipe = ((command_no - 1) % 2) * 2;
        int current_pipe = (command_no % 2) * 2;

        // If current command is the first command, close the
        // read end, else read from the last command's pipe
        if(command_no == 0)
        {
            close(pipefd[0]);
        }
        else
        {
            dup2(pipefd[prev_pipe], 0);
            close(pipefd[current_pipe]);
        }

        // If current command is the last command, close the
        // write end, else write to the pipe
        if(command_no == n_commands - 1)
        {
            close(pipefd[current_pipe + 1]);
        }
        else
        {
            dup2(pipefd[current_pipe + 1], 1);
        }

        int p = execvp(tokens[cmd_pos[command_no]], tokens + cmd_pos[command_no]);

        close(pipefd[current_pipe]);
        close(pipefd[prev_pipe]);
        close(pipefd[prev_pipe + 1]);
        close(pipefd[current_pipe + 1]);

        _exit(0);
    }
}

It seems that the programs from /usr/bin are not being executed if they are not the first command in the pipeline.

Answer 1

You are connecting the pipes incorrectly.

This logic:

int prev_pipe = ((command_no - 1) % 2) * 2;
int current_pipe = (command_no % 2) * 2;

doesn't work — the result of modulo will always be 0 or 1 , so prev_pipe and current_pipe will be either 0 or 2 ...

Well, unless I'm missing some hidden concept because you didn't paste any code creating the pipes.

Answer 2

Here's a very simple program created from your code — guessing how pipes might be created and simplifying the command argv handling a bit:

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

static char *argv_ls[] = { "ls", 0 };
static char *argv_wc[] = { "wc", 0 };
static char **cmds[]   = { argv_ls, argv_wc };

int main(void)
{
    int n_commands = 2;
    int pipefd[2];

    pipe(&pipefd[0]);   // Error check!

    fflush(stdout);
    for (int i = 0; i < n_commands; i++)
    {
        int pid = fork();

        if (pid == 0)
        {
            int command_no = i;
            int prev_pipe = ((command_no - 1) % 2) * 2;
            int current_pipe = (command_no % 2) * 2;
            printf("cmd %d: prev pipe %d, curr pipe %d\n", i, prev_pipe, current_pipe);
            fflush(stdout);

            // If current command is the first command, close the
            // read end, else read from the last command's pipe
            if (command_no == 0)
            {
                close(pipefd[0]);
            }
            else
            {
                dup2(pipefd[prev_pipe], 0);
                close(pipefd[current_pipe]);  // Line 40
            }

            // If current command is the last command, close the
            // write end, else write to the pipe
            if (command_no == n_commands - 1)
                close(pipefd[current_pipe + 1]);  // Line 46
            else
                dup2(pipefd[current_pipe + 1], 1);

            execvp(cmds[i][0], cmds[i]);
            fprintf(stderr, "Failed to exec: %s (%d: %s)\n", cmds[i][0], errno, strerror(errno));
            _exit(1);
        }
    }

    return 0;
}

When GCC 4.7.1 (on Mac OS X 10.7.4) compiles it, it warns:

pipes-12133858.c: In function ‘main’:
pipes-12133858.c:40:22: warning: array subscript is above array bounds [-Warray-bounds]
pipes-12133858.c:46:22: warning: array subscript is above array bounds [-Warray-bounds]

When I run it, I get the output:

Isis JL: pipes-12133858
cmd 0: prev pipe -2, curr pipe 0
cmd 1: prev pipe 0, curr pipe 2
Isis JL: wc: stdin: read: Bad file descriptor

Since the parent in the code does not wait for the children to finish, the prompt appears before the error message from wc , but the diagnostic numbers printed show that there are all sorts of problems (and the compiler was able to spot some of the problems).

Note that there is no need to check the return value from any of the exec*() family of functions. If they are successful, they do not return; if they return, they failed. There was also no need for the closes before calling _exit(0); since the system will close them anyway. Also, when you fail to exec something, it is courteous to print a message indicating what you failed to execute and to exit with a non-zero exit status.

So, as Michał Górny says, a major part of your problem is that your pipe handling code is at least mysterious because you didn't show it and probably erroneous.

I'm also tolerably sure you don't have enough close() calls in your code. As a guideline, in each process that has pipes open in it that is going to be part of a pipeline, all the file descriptors returned by the pipe() system call should be closed before any given child process uses an exec*() function. Not closing pipes can lead to processes hanging because the write end of a pipe is open. If the process that has the write end open is the process that's trying to read from the read end of the pipe, then it isn't going to find any data to read.