Multiple tests with Pipes IPC between parent and child in C

Question

I am writing a program of Parented Process Communication with pipes, the point is that I need to send from an endline different amounts of data (large ones) but I don't know how to ordered 'split' the message into smaller ones (4096 Bytes each one) and send them in order to the read endline. I've tried with a cycle that makes (message size / 4096) iterations but it didn't work so I let that idea away.

Here is my code:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <string.h>
#include <fcntl.h>
#include <time.h>

#define SIZES 6
#define KB 1024

//1KB, 10KB, 100KB, 1MB, 10MB, 100MB
int PACK_SIZES[SIZES] = {1*KB, 10*KB, 100*KB, 1*KB*KB, 10*KB*KB, 100*KB*KB};
//int PACK_TESTS[SIZES] = {1000,1000,100,100,10,5};
int PACK_TESTS[SIZES] = {6,5,4,3,2,1};

void fill(char * bufptr, size_t size);

int main() {
    int tests_amount;
    size_t size;
    
    //pipefd[0] -> lectura
    //pipefd[1] -> escritura
    int pipefd[2];

    int r;  //Variable de retorno.
    pid_t pid;

    r = pipe(pipefd);
    if (r < 0){
        perror("Error al crear la tubería");
        exit(-1);
    }

    printf("Medidas de envío de paquetes con comunicación mediante Tuberías.\n\n");

    pid = fork();
    if (pid == (pid_t) -1){
        perror("Error en fork");
        exit(-1);
    }

    for (int i = 0; i < SIZES; i++){
        int packages_amount;
        tests_amount = PACK_TESTS[i];
        size = PACK_SIZES[i];

        float time0, time1;
        float total_time = 0;

        char *write_buffer[size];
        char *read_buffer[size];
        fill(write_buffer, size);

        for (int j = 1; j <= tests_amount; j++){
            time0 = clock();

            if (pid == 0){
                //Consumidor

                close(pipefd[1]);
                read(pipefd[0], read_buffer, size);
                close(pipefd[0]);
                //printf("\n Mensaje recibido de tamaño %d: %d \n", size, strlen(read_buffer));

            }else {
                //Productor
                close(pipefd[0]);
                write(pipefd[1], write_buffer, size);
                close(pipefd[1]);
            }
            time1 = clock();
            double tiempo = (double)(time1) / CLOCKS_PER_SEC-(double)(time0) / CLOCKS_PER_SEC;
            total_time = total_time + tiempo;
        }

        printf("El promedio con %d Bytes y %d pruebas es: %f seg. \n\n",size,tests_amount,total_time/tests_amount);
        total_time = 0;

    }
    return 0;
}

void fill(char * bufptr, size_t size){
    static char ch = 'A';
    int filled_count;

    //printf("size is %d\n", size);
    if (size != 0){
        memset(bufptr, ch, size);
    } else {
        memset(bufptr, 'q', size);
    }
    
    bufptr[size-1] = '\0';
    
    if (ch > 90)
        ch = 65;
    
    filled_count = strlen(bufptr);

    //printf("Bytes escritos: %d\n\n", filled_count);
    //printf("buffer filled is:%s\n", bufptr);
    ch++;
}

I get something like the following error (I know it's race conditioned but I don't know how to fix it :c):

Medidas de envío de paquetes con comunicación mediante Tuberías.

El promedio con 1024 Bytes y 6 pruebas es: 0.000002 seg. 


 Mensaje recibido de tamaño 1024: 1023 

 Mensaje recibido de tamaño 1024: 1023 

 Mensaje recibido de tamaño 1024: 1023 

 Mensaje recibido de tamaño 1024: 1023 

 Mensaje recibido de tamaño 1024: 1023 

 Mensaje recibido de tamaño 1024: 1023 
El promedio con 10240 Bytes y 5 pruebas es: 0.000001 seg. 

El promedio con 1024 Bytes y 6 pruebas es: 0.000012 seg. 


 Mensaje recibido de tamaño 10240: 0 

 Mensaje recibido de tamaño 10240: 0 

 Mensaje recibido de tamaño 10240: 0 

 Mensaje recibido de tamaño 10240: 0 

 Mensaje recibido de tamaño 10240: 0 
El promedio con 10240 Bytes y 5 pruebas es: 0.000005 seg. 

El promedio con 102400 Bytes y 4 pruebas es: 0.000001 seg. 


 Mensaje recibido de tamaño 102400: 0 

 Mensaje recibido de tamaño 102400: 0 

 Mensaje recibido de tamaño 102400: 0 

 Mensaje recibido de tamaño 102400: 0 
El promedio con 102400 Bytes y 4 pruebas es: 0.000008 seg. 

Segmentation fault (core dumped)

Answer 1

There are two three issues with

char *write_buffer[size];
char *read_buffer[size];

Firstly, these are variable-length arrays allocated with automatic storage duration . This is also referred to as being "allocated on the stack" . Most operating systems have different limits placed on how large a program's stack can be. In a modern Linux, this is usually defaults to 8MB ( ulimit -s to check).

With the given sizes, you are easily going to overflow the stack on most machines.

Secondly, these are arrays of pointer-to-char , meaning the space required is multiplied by sizeof (char *) , and these are not the correct types to store null-terminated strings.

Turning up your compiler's warning levels should have alerted you to the second problem, with warnings such as

main.c:55:14: warning: passing argument 1 of ‘fill’ from incompatible pointer type [-Wincompatible-pointer-types]
   55 |         fill(write_buffer, size);
      |              ^~~~~~~~~~~~
      |              |
      |              char **
main.c:16:17: note: expected ‘char *’ but argument is of type ‘char **’
   16 | void fill(char *bufptr, size_t size);
      |           ~~~~~~^~~~~~

and similarly for strlen .

Thirdly, both buffers exist in both processes, wherein each process only uses one.

Although this is a toy program, the use of such large buffers is dubious. You are dealing with very simple fixed-length messages. The writer is writing the same byte size - 1 times, followed by the zero byte. The reader can just consume size bytes.

---

The next issue is that you repeatedly close both ends of the pipe in each iteration of the inner loop, in each process.

The reading process should close the write end of the pipe, just once, before the outer loop. The writing process should close the read end of the pipe, just once, before the outer loop.

After the loop, both processes should close their side of the pipe.

---

You do not check the return values of read or write to confirm an error did not occur, or that the amount of data expected to be processed was done so fully.

---

fill is flawed, although not seen to be so with the way it is used.

if (size != 0){
    memset(bufptr, ch, size);
} else {
    memset(bufptr, 'q', size);
}

bufptr[size-1] = '\0';

If size is zero:

• memset(bufptr, 'q', size); is effectively a NOP , having no effect.

• bufptr[size-1] = '\0'; will cause unsigned integer overflow, and will index the offset in the array of SIZE_MAX .

Note that if (ch > 90) is only checked after ch has been used. With ASCII, the 27th use of this function will yield a string containing [ .

filled_count is rather pointless. The string length will be size - 1 .

---

Here is a cursory example program to play around with.

If you are interested in profiling the difference between processing a single byte at a time, and processing larger buffers, or want to actually keep the data you process, and want to reintroduce string buffers, know that char foo[size]; will really only work for relatively small values of size . You can try using dynamic memory instead.

#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <time.h>
#include <unistd.h>

#define SIZES 6
#define KB 1024

/* 1KB, 10KB, 100KB, 1MB, 10MB, 100MB */
size_t PACK_SIZES[SIZES] = {1*KB, 10*KB, 100*KB, 1*KB*KB, 10*KB*KB, 100*KB*KB};
int PACK_TESTS[SIZES] = {6,5,4,3,2,1};

static void pdie(const char *msg) {
    perror(msg);
    exit(EXIT_FAILURE);
}

static char get_next_byte(void) {
    static char byte = 'A';

    if (byte > 'Z')
        byte = 'A';

    return byte++;
}

static int read_exactly(int fd, size_t size) {
    unsigned char b;

    while (size > 0 && 1 == read(fd, &b, 1))
        size--;

    return 0 == size;
}

static int write_exact_string(int fd, size_t size, char b) {
    unsigned char zero = 0;

    while (size > 1 && 1 == write(fd, &b, 1))
        size--;

    return (1 == size) && (1 == write(fd, &zero, 1));
}

int main(void) {
    pid_t pid;
    int pipefd[2];

    if (-1 == pipe(pipefd))
        pdie("pipe");

    if (-1 == (pid = fork()))
        pdie("fork");

    if (0 == pid)
        close(pipefd[1]);
    else
        close(pipefd[0]);

    for (int i = 1; i <= SIZES; i++) {
        size_t expected_size = PACK_SIZES[i];
        int tests_amount = PACK_TESTS[i];

        double total_time = 0;

        for (int j = 1; j <= tests_amount; j++) {
            clock_t start = clock();

            if (pid == 0) {
                if (!read_exactly(pipefd[0], expected_size)) {
                    fprintf(stderr, "Failed to read %zu bytes.\n", expected_size);
                    exit(EXIT_FAILURE);
                }
            } else {
                char byte = get_next_byte();

                if (!write_exact_string(pipefd[1], expected_size, byte)) {
                    fprintf(stderr, "Failed to write an exact string of [%zu*'%c'+'\\0'].\n", expected_size - 1, byte);
                    exit(EXIT_FAILURE);
                }
            }

            double duration = ((double) (clock() - start)) / CLOCKS_PER_SEC;

            printf("Test<%d-%d>: %s %zu bytes in %f.\n", i, j, (pid == 0) ? "Read" : "Wrote", expected_size, duration);
            total_time += duration;
        }

        printf("Time taken in <%s> to process %d * %zu bytes: %f (%f avg.)\n",
                (pid == 0) ? "CHILD" : "PARENT",
                tests_amount,
                expected_size,
                total_time,
                total_time / tests_amount);
    }

    if (0 == pid) {
        close(pipefd[0]);
    } else {
        close(pipefd[1]);

        if (-1 == waitpid(pid, NULL, WUNTRACED))
            pdie("waitpid");
    }
}