Subtract two elements' (from the same structure) addresses?

Question

I am writing a program and I need to calculate the difference between two addresses of a structure's elements. Sorry if it's not clear, some code might be more explicit.

There are some MPI elements in the code but it's not what my problem's about.

typedef struct Str {
  std::vector<int> itab;
  std::vector<char> ctab;
} s_str;

int main(int argc, char **argv)
{
  s_str *struc1 = new s_str();

  MPI_Datatype Strtype;
  int count = 2;
  int lengths[2] = { 8, 8 };
  MPI_Aint disp[2];
  disp[0] = &struc1->itab - &struc1;
  disp[1] = &struc1->ctab - &struc1;
  MPI_Datatype types[2] = { MPI_INT, MPI_CHAR };
  MPI_Type_create_struct(count, lengths, disp, types, &Strtype);
  MPI_Type_commit(&Strtype);

  (...)
}

The problem takes place at these lines :

MPI_Aint disp[2];
disp[0] = &struc1->itab - &struc1;
disp[1] = &struc1->ctab - &struc1;

First of all, MPI_Aint is a "C type that holds any valid address" (taken from the documentation). What I'm trying to do here, it calculate the difference between struc1->itab and struc1 addresses.

Here are the errors I'm getting :

tmp2.cpp: In function ‘int main(int, char**)’:
tmp2.cpp:30:30: erreur: invalid operands of types ‘std::vector<int>*’ and ‘s_str** {aka Str**}’ to binary ‘operator-’
tmp2.cpp:31:30: erreur: invalid operands of types ‘std::vector<char>*’ and ‘s_str** {aka Str**}’ to binary ‘operator-’

I guess I'm not correctly accessing the addresses, since the types involved here are std::vector<int>* and s_str** . What's the proper way ?

More details : I need this to write in a file using MPI-I/O. It's to tell MPI how many space each proc will need to write its whole dataset. So basically, I just want to write the content of a user-defined struct in a file, using MPI-I/O. Here's why.

Answer 1

I think your reasoning is a bit wrong, even if you give the address of where the vectors are the actual elements of the vector may be placed elsewhere.

try for instance eg

s_str struc1;
struc1.itab.push_back(0);
printf( "struc1.itab %p\n", (void*)&(struc1.itab) );
printf( "struc1.ctab %p\n", (void*)&(struc1.ctab) );
printf( "struc1.itab[0] %p\n", (void*)&(struc1.itab[0]) );
printf( "struc1 %p\n", (void*)&struc1 );

struc1.itab 000000000022fde0
struc1.ctab 000000000022fdf8
struc1.itab[0] 0000000000327f40
struc1 000000000022fde0

maybe you had originally a struct that had two normal c-arrays?

typedef struct 
{
  int itab[100];
  char ctab[100];
} struc1;

then it would make more sense what you are doing.

Answer 2

try this:

 disp[0] = ((char *) &(struc1->itab)) - ((char *) struc1);
 disp[1] = ((char *) &(struc1->ctab)) - ((char *) struc1);

Answer 3

The portable MPI way would be:

MPI_Aint base;

MPI_Get_address(&struc1, &base);
MPI_Get_address(&struc1->itab, &disp[0]);
disp[0] -= base;
MPI_Get_address(&struc1->ctab, &disp[1]);
disp[1] -= base;

Anyway, this won't work because std::vector allocates memory for its elements on the heap. As heap allocations might end up in completely random locations in the process' address space, one should use a slightly different approach:

MPI_Aint disp[2];

// N.B. both displacements are ABSOLUTE addresses
MPI_Get_address(&struc1->itab[0], &disp[0]);
MPI_Get_address(&struc1->ctab[0], &disp[1]);

MPI_Type_create_struct(count, lengths, disp, types, &Strtype);
MPI_Type_commit(&Strtype);

MPI_Send(MPI_BOTTOM, 1, Strtype, ...);

MPI_Type_free(&Strtype);

The structure type is constructed using absolute addresses and therefore MPI_BOTTOM must be used in the MPI_Send call ( MPI_BOTTOM signifies that the buffer address is the bottom of the process' address space). This only works for sending one instance of the structure type, therefore the type is destroyed immediately after the send call. In principle, creating a new datatype for each structure instance to be sent is not a big problem since sending data is much more expensive operation than type creation and destruction.

Answer 4

This is what I used. It works on my project, I don't know why both the solutions work though, since I don't think they have the same meaning :

MPI_Aint disp[2] = { 0, (uintptr_t)&(struc1->ctab[0]) - (uintptr_t)&(struc1->itab[0]) };
MPI_Aint disp[2] = { 0, sizeof (std::vector<int>) + sizeof (std::vector<char>) };
// Why do both of these solutions work ?

Could someone confirm that this is not specific to my case and that those methods are not completely strange/bad?