link cython module in a c++ program

Question

Is it possible to build a cython module with some cdef functions and link the resulting shared library into a C++ program?

I tried a proof of concept:

cymod.pyx:

# distutils: language=c++

from libcpp.string cimport string

cdef public string simple_echo(string test_string):
    return test_string

cpp_test.cpp:

#define PyMODINIT_FUNC void
#include <iostream>
#include "cymod.h"

int main(int argc, char const *argv[])
{
    std::cout << simple_echo("test") << std::endl;
    return 0;
}

setup.py:

from setuptools import setup, Extension
from Cython.Build import cythonize

setup(
    name='cymod',
    ext_modules=cythonize(
        Extension(
            "cymod", ["cymod.pyx"],
        ),
    )
)

The cython module builds fine, but when I try to build the c++ code that will use the cython function I get:

$ g++ -L. -l:cymod.so cpp_test.cpp -o cpp_test
/tmp/cc48Vc2z.o: In function `main':
cpp_test.cpp:(.text+0x51): undefined reference to `simple_echo'
collect2: error: ld returned 1 exit status

Which is odd. The generated header file has it:

cymod.h:

  /* Generated by Cython 0.29.1 */

  #ifndef __PYX_HAVE__cymod
  #define __PYX_HAVE__cymod


  #ifndef __PYX_HAVE_API__cymod

  #ifndef __PYX_EXTERN_C
    #ifdef __cplusplus
      #define __PYX_EXTERN_C extern "C"
    #else
      #define __PYX_EXTERN_C extern
    #endif
  #endif

  #ifndef DL_IMPORT
    #define DL_IMPORT(_T) _T
  #endif

  __PYX_EXTERN_C std::string simple_echo(std::string);

  #endif /* !__PYX_HAVE_API__cymod */

  /* WARNING: the interface of the module init function changed in CPython 3.5. */
  /* It now returns a PyModuleDef instance instead of a PyModule instance. */

  #if PY_MAJOR_VERSION < 3
  PyMODINIT_FUNC initcymod(void);
  #else
  PyMODINIT_FUNC PyInit_cymod(void);
  #endif

  #endif /* !__PYX_HAVE__cymod */

and I see my function in cymod.so :

nm cymod.so| grep simple_echo
0000000000001e50 T simple_echo

NOTE: I realize that to actually get this working I'll need to link against the python libraries and initialize the interpreter etc. I left that out to make this a tad shorter and I get the same error either way.

Answer 1

The short answer is that I was putting the -l argument too early in the compilation command. It is also important to handle the library lookup path. The simplest way is to use rpath . I set the rpath to the directory that the executable is in, ie, .

Additionally, it is necessary to link against the python libraries and set the include and library paths. These can be determined at compile time by using the output of the python-config utility. Here is the compilation command that ultimately did the trick:

g++ cpp_test.cpp -o cpp_test -L. -l:cymod.so $(python-config --libs) $(python-config --includes) $(python-config --cflags) -Wl,-rpath,"\$ORIGIN"

I also updated the c++ file to #include "Python.h" and added calls to Py_Initialize() , Py_Finalize() , and initcymod() :

#include <iostream>
#include "Python.h"
#include "cymod.h"

int main(int argc, char *argv[])
{
    Py_Initialize();
    initcymod();
    std::cout << simple_echo("test") << std::endl;
    Py_Finalize();
    return 0;
}

NOTE: the call to initcymod() is necessary, but python2 specific. On python3 you should call PyImport_AppendInittab("cymod", PyInit_cymod); prior to Py_Initialize() . The cymod part is the module name, substitute your module name.

Thanks to @ead for the informative link to the docs on this topic https://cython.readthedocs.io/en/latest/src/userguide/external_C_code.html#using-cython-declarations-from-c and his answer to a related question https://stackoverflow.com/a/45424720/2069572

While reading the linked docs, I came across this:

Note On some operating systems like Linux, it is also possible to first build the Cython extension in the usual way and then link against the resulting .so file like a dynamic library. Beware that this is not portable, so it should be avoided.

So it turns out that you should not do what I was trying to do.

Instead, what I should have done was run:

cython --cplus cymod.pyx

And then compiled cpp_test.cpp with the generated cymod.cpp file. No need to link the cython shared library, and it turns out that it is not a good idea to do so.