Strange process fail with boost.python

Question

I use boost.python for calling script. Each python script has 3 functions:

• Initialize - that have to be called at start
• Uninitialize - that have to be called at finish for closing every object.
• Process - that have to be called multiple times between Initialize and Uninitialize.

This class don't need in explanation:

class PyInit
{
public:
    PyInit() { Py_Initialize(); }
    ~PyInit() {  Py_Finalize();  }
};

This class creates PyInit object instance, then initialize all Python objects and makes call to "Initialize()" python function at constructor. And calls "Uninitialize()" at destructor.

And this class has function "Process" to call it outside of class multiple times.

class PyGuard
{
public:
    explicit PyGuard(const char* pypath)
    {
      try
      {
        _main_module = boost::python::import("__main__");
        _main_namespace = _main_module.attr("__dict__");
        boost::python::object ignored = exec_file(pypath, _main_namespace);
        _Initialize = _main_namespace["Initialize"];
        _Process = _main_namespace["Process"];
        _Uninitialize = _main_namespace["Uninitialize"];

        _InitRes = _Initialize();
        _ProcRes = boost::python::object();
      }
      catch(boost::python::error_already_set&)
      {
        string res;
        py_utils::err_parse(res);
        throw string("Python exception: " + res);
      }
      catch(std::exception& e)
      {
          throw string("C++ exception: " + string(e.what()));
      }
      catch(...)
      {
          throw string("Unhandled exception!");
      }
    }
    virtual ~PyGuard()
    {
        _Uninitialize(_InitRes, _ProcRes);
    }
    void Process()
    {
        _ProcRes = _Process(_InitRes, _ProcRes);
    }


private:
    py_utils::PyInit _initPython;

    boost::python::object _InitRes;
    boost::python::object _ProcRes; 

    boost::python::object _Process;
    boost::python::object _Initialize;
    boost::python::object _Uninitialize;

    boost::python::object _main_module;
    boost::python::object _main_namespace;
};

As only "Process" fails with "boost::python::error_already_set" exception, current instance of PyGuard is deleted and then created new one. To avoid hidden dependencies that could be as hidden exception result.

So, after exception, all python stuff is removed (even calls Py_Finalize()), and then created again.

Still, after 4-10 such exceptions, whole c++ process falls.

Falls, avoiding even catchers:

  try
  {
      _PyGuard = make_shared<PyGuard>("my script");
      while(true)
      {
        try {
          _PyGuard->Process();
        }
        catch()
        {
              bool inited = false;
              while(!inited)
              {
                try
                {
                _pyGuard = nullptr;
                _pyGuard = make_shared<PyGuard>("script path.txt");
                inited = true;
                }
                catch(string& e)
                {
                }
              }
        }
      } 
      _PyGuard = nullptr;
  } 
  catch(...)
  {
      //falling, it never being catched here.
  }

So, my question is, why can it falls and being uncatchable?

I just found that it falls at line:

_pyGuard = make_shared<PyGuard>("script path.txt"); 

call, so I feel it is Python raise exception that can't be catched. Why? And how to prevent it?

Answer 1

It's possible that your use of Py_Finalize() is the problem. According to the Boost 1.55 documentation, you shouldn't use Py_Finalize() with Boost.Python . It doesn't look like your program really needs finalization, so you might try just removing the call from ~PyInit() .

If you really do need finalization for some reason, you might look at Py_NewInterpreter() .

As for your "uncatchable exceptions" problem, that's often an effect of having two active exceptions at once. When that happens, C++ will simply abort. That may (or may not) be what's happening with your code.