Allocating memory buffer for use in MPI_Pack()

Question

I'm going to use MPI_Pack() to make a message composed of n int s and m double s. Their positions in the message buffer will be something like this

p1 x int s, q1 x double s, p2 x int s, q2 x double s, ..., pN x int s, qN x double s

where n=p1+p2+...+pN and m=q1+q2+...+qN.

My question: Is the size of this message equal to the size of a message composed of the same number of int s and double s but with the following order:

nx int s, mx double s

I'm asking this question because I want to know how much memory should be allocated for the buffer. If the size of the message depends only on the number of int s and double s and not how they are arranged, then the buffer can be allocated very easily:

MPI_Pack_size(n, MPI_INT, communicator, &k1);
MPI_Pack_size(m, MPI_DOUBLE, communicator, &k2);
buffer = malloc(k1 + k2);

Obviously the following solution is correct:

k = 0;
for (int i=0; i < N; i++)
{
    MPI_Pack_size(p[i], MPI_INT, communicator, &k1);
    MPI_Pack_size(q[i], MPI_DOUBLE, communicator, &k2);
    k += k1 + k2;
}
buffer = malloc(k);

But for a large N, it may result in a too excessively large buffer, because as the official document of MPI states, the routine MPI_Pack_size()

returns an upper bound, rather than an exact bound, since the exact amount of space needed to pack the message may depend on the context (eg, first message packed in a packing unit may take more space).

UPDATE: a program I wrote for testing if the order of packing the int s and double s affect the size of the message.

#include <stdio.h>
#include <mpi.h>
#include <assert.h>
#include <stdlib.h>
#include <time.h>

#define BUFF_SIZE 200000   /* buffer size in bytes */
#define MY_MPI_REAL MPI_DOUBLE

typedef double real;

int main()
{
    MPI_Init(NULL, NULL);

    int ic = 0, rc = 0; /* counters of int and real numbers */
    int pos = 0;   /* position in the buffer, used in MPI_Pack() calls */

    /* allocate memory of the pack buffer */

    void *buff = malloc(BUFF_SIZE);
    assert(buff);

    /* case 1: packing a large number of pairs of arrays */

    srand(time(NULL));

    for (int i=0; i<100; i++)  /* 100 array pairs */
    {
        /* make int and real arrays of random lengths */
        int ik = 99 * ((double)rand() / RAND_MAX) + 1;
        int rk = 99 * ((double)rand() / RAND_MAX) + 1;
        int *iarr = (int *)malloc(ik * sizeof(int));
        assert(iarr);
        double *rarr = (real *)malloc(rk * sizeof(real));
        assert(rarr);

        ic += ik;
        rc += rk;

        /* pack the array pair */
        MPI_Pack(iarr, ik, MPI_INT, buff, BUFF_SIZE, &pos, MPI_COMM_WORLD);
        MPI_Pack(rarr, rk, MY_MPI_REAL, buff, BUFF_SIZE, &pos, MPI_COMM_WORLD);

        free(iarr);
        free(rarr);
    }

    printf("final position for case 1 = %d\n", pos);

    /* case 2: packing a single pair of arrays */

    pos = 0;

    int *iarr = (int *)malloc(ic * sizeof(int));
    assert(iarr);
    double *rarr = (real *)malloc(rc * sizeof(real));
    assert(rarr);

    MPI_Pack(iarr, ic, MPI_INT, buff, BUFF_SIZE, &pos, MPI_COMM_WORLD);
    MPI_Pack(rarr, rc, MY_MPI_REAL, buff, BUFF_SIZE, &pos, MPI_COMM_WORLD);

    free(iarr);
    free(rarr);

    printf("final position for case 2 = %d\n", pos);

    free(buff);

    printf("sizeof(int) = %ld, sizeof(real) = %ld\n", sizeof(int), sizeof(real));
    printf("num of ints = %d, num of reals = %d\n", ic, rc);
    printf("num of ints x sizeof(int) + num of reals x sizeof(real) = %ld\n", ic*sizeof(int)+rc*sizeof(real));

    MPI_Finalize();
}

Answer 1

I think your worries are misplaced. The only possible overhead I see would be from alignment: maybe a one time alignment at the start of the buffer, and then maybe per element. However, the pack buffer is counted in bytes, and I just tested it: even packing a single byte does not lead to any padding. So that leads me to suspect that every data type basically takes the exact amount of space.