How to debug with LD_preload

Question

I use LD_PRELOAD to override the MPI_Send function with my own function to do some debugging of the MPI_send function.

Here, myMPI_Send.c code:

#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
#include <mpi.h>
#include <stdlib.h>

int MPI_Send(const void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm)
{
        int (*original_MPI_Send)(const void *, int, MPI_Datatype, int, int, MPI_Comm);    
        original_MPI_Send=dlsym(RTLD_NEXT, "MPI_Send");    
        printf(" Calling MPI_Send ************** \n");    
        return (*original_MPI_Send)(buf, count, datatype, dest, tag, comm);
}

In my project, I use an extern library which includes also MPI_Send functions. I need to debug the extern library to know the line and the number of calls of each call of MPI_Send.

I tried to use this code using macros:

fprintf (stderr,"MPI_Send, func <%s>, file %s, line %d, count %d\n",__func__, __FILE__, __LINE__, __COUNTER__);

But, it doesn't work, it prints always the line of MPI_Send in the myMPI_Send.so.

Could you help me please? Thank you in advance.

Answer 1

MPI covers most of your needs via the MPI Profiling Interface (aka PMPI).

Simply redifines the MPI_* subroutines you need, and have them call the original PMPI_* corresponding subroutine.

In you case:

int MPI_Send(const void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm)
{
        printf(" Calling MPI_Send ************** \n");    
        PMPI_Send(buf, count, datatype, dest, tag, comm);
}

Since you want to print the line and file of the caller, you might have to use macros and rebuild your app:

#define MPI_Send(buf,count,MPI_Datatype, dest, tag, comm) \
    myMPI_Send(buf, count, MPI_Datatype, dest, tag, comm, __func__, __FILE__, __LINE__)

int myMPI_Send(const void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm, char *func, char *file, int line)
{
        printf(" Calling MPI_Send ************** from %s at %s:%d\n", func, file, line);    
        return PMPI_Send(buf, count, datatype, dest, tag, comm);
}