如何从Python调用MPI .so文件？

Question

我有一个共享的目标文件，其中包含已编译的C ++ MPI Hello World代码。 当我尝试使用ctypes从Python调用它时，我得到了一些无用的错误列表。

mpiHello.cpp：

#include <mpi.h>

extern "C"
void mpiHello() {

    int rank, size;

    MPI_Init(NULL, NULL);

    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &size);

    std::cout << "Hello world! I am " << rank << " of " << size << std::endl;

    MPI_Finalize();

}

编译命令： mpic++ -fPIC -o mpi.so mpiHello.cpp -shared

Python调用：

    def __init__(self):
        self.dll = None
        _DIRNAME = os.path.dirname(__file__)
        try:  # Windows
            self.dll = ctypes.CDLL(os.path.join(_DIRNAME, "mpi.dll"))
        except OSError:  # Linux
            self.dll = ctypes.cdll.LoadLibrary(os.path.join(_DIRNAME, "mpi.so"))
        finally:
            self.dll.mpiHello.argtypes = []

    def execute(self):
        self.dll.mpiHello()

_mpi = mpi()
_mpi.execute()

[<user>-OptiPlex-7050:09468] mca_base_component_repository_open: unable to open mca_shmem_mmap: /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[<user>-OptiPlex-7050:09468] mca_base_component_repository_open: unable to open mca_shmem_sysv: /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[<user>-OptiPlex-7050:09468] mca_base_component_repository_open: unable to open mca_shmem_posix: /usr/lib/x86_64-linux-gnu/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[<user>-OptiPlex-7050:9468] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!

Process finished with exit code 1

我希望代码显示4行并显示“ Hello world！我是...但是我只是收到错误消息。任何帮助将不胜感激！

Answer 1

在Linux上几乎没有什么可做的，实际上您的错误缺少括号。

#include "mpiHello.h" 

extern "C" {
    void mpiHello() {
        // Code
    }
}

与mpiHello.h文件相同：

#pragma once
#include <mpi.h>

extern "C" {
    void mpiHello();
}

与python使用共享库。 我通常使用宏来处理导出范围，它将与Windows（ .dll ）和Linux（ .so ）一起使用。

mpi.cpp ：

#include "mpiHello.h"

extern "C" {
    MYEXPORT void mpiHello() {
        // Code
    }
}

mpiHello.h ：

#pragma once
#include <mpi.h>
#if _WIN32
#define MYEXPORT __declspec(dllexport)
#else
#define MYEXPORT 
#endif

extern "C" {
    MYEXPORT void mpiHello();
}

如果要使用返回值或参数创建新功能，请小心。 您将需要在python文件中指定它们。

如果您有新的int myFonction(char* str1, char* str2, int pos) ，那么您将在python中有以下声明：

dll.myFonction.argtypes = [ctype.c_char_p, ctype.c_char_p, ctype.c_int]
dll.myFunction.restype = ctype.c_int

此外，您将必须使用C类型参数来提供新功能，因此必须将python值转换为C值。 这是将python列表和python int转换为ctypes的示例：

pyhton_string = "Hello Word"
python_int = 42

c_string = (ctype.c_char* len(pyhton_string))(*pyhton_string)
c_int = ctype.c_int(python_int)