creating static library from Python code on Ubuntu

Question

Context: I want to use some of the Python code that I wrote and call it from Matlab (not an ideal set up, but that's the constraint I have to live with for now). The code uses numpy, scipy, and pandas packages. I have tried dynamically linking options and got some success with matpy . However, to use numpy I need to override Matlab's own libraries (libgfortran, liblapack, libblas, libstdc++) and instead point to system's version of the same libraries using LD_PRELOAD. Then if Matlab code downstream tries to use these libs, it invariably segfaults. This is turning out to be less reliable, and even if I fix all the issues with this now, it is prone to unpredictable consequences later on as the downstream Matlab code evolves.

Hence I have decided to compile python code into a static lib (including all dependencies from numpy, scipy and pandas) , and then write a Mex that links against this static lib.

In that context, I want your expert opinion on following questions:

1. Which are the most reliable tools for compiling the python code into a STATIC lib with the goal of then linking it into a Mex? I am for now only looking at Ubuntu 12.04 platform. I looked at Cython, Pyrex, pyInstaller, py2exe, cx_freeze -- great tools, but being a newbie to python and not having compiled python into anything but the automated .pyc, I need your help in making this choice.

2. What are the pitfalls / things that could go wrong / things I should be careful about in this set up? I am referring to (custom.py + numpy + scipy + pandas) --> static lib ---> Mex -->called from Matlab setup.

Thanks a ton!

Answer 1

I suggest you communicate between the Mex and Python using a socket instead of calling the Python code directly. The OS will do the the socket transfer using a memory-to-memory (or mapping tricks) so it will go fast. Once you've got separate executables, you should be able to get out of this shared library hell.

If you still need Python to be a single executable, consider pyinstaller or cx_freeze .