Running several independent instances of python pyd module

Question

I wondering is there a solution to import one compiled from C++ source pyd module twice. So to have 2 separate instances of one module with distinct values of variables defined in C? Here is the example. I have a simple cpp module:

#include <boost/python/module.hpp>
#include <boost/python/def.hpp>
int n;
void set(int i)
{
    n = i;
}
int get()
{
    return n;
}
BOOST_PYTHON_MODULE(test_ext)
{
    using namespace boost::python;
    def("set", set);
    def("get", get);
}

now I'm trying to import it twice:

In [1]: import sys

In [2]: import test_ext as t1

In [3]: del sys.modules['test_ext']

In [4]: import test_ext as t2

In [5]: t1.set(1)

In [6]: t2.set(2)

In [7]: t1.x=1

In [8]: t2.x=2

In [9]: t1.x
Out[9]: 1

In [10]: t2.x
Out[10]: 2

In [11]: t1.get()
Out[11]: 2

In [12]: t2.get()
Out[12]: 2

As you can see both modules points to the same variable. If I setting it in one module it changed in another.

Actually I have a code generated by Matlab and there are many global variables in it. I want to find a way to run this code independently in several instances of module. By the way I'm using python 2.7

Thanks in advance!

Answer 1

An issue with this is that C extensions of Python are escaping Python VM sandbox (there is even note of this behaviour in Python documentation).

You virtually cannot expect to have more than one instance of global variable in C in one address space (eg one process). It is just not possible and it is even not anyhow connected to Python.

BTW: You can actually even 'interconnect' two Python VMs running in single process this way.

So how to solve this:

1. C-level global variables are actually a map (or dictionaries) where key is identifier of Python VM (or anything else that will fit your case). Since you are in C, this is however not the easiest case.

2. Encapsulate module in dedicated process space - start a subprocess. You will need to however resolve how to communicate with parent process if needed.