Call multiple python functions from different directories

Question

I have some code that will go to a directory (Folder 1 for demonstration purposes), then call a function called function in the file python_function.py . The code looks like this:

#include <Python.h>
#include <string>
#include <iostream>

int main()
{
    PyObject *pName, *pModule, *pDict, *pFunc;

    setenv("PYTHONDONTWRITEBYTECODE", " ", 1);

    // Initialize the Python Interpreter
    Py_Initialize();

    //CALL FUNCTION FROM FOLDER 1:
    std::wstring pathWide = L"./Folder 1";
    PySys_SetPath(pathWide.c_str());

    // Build the name object
    pName = PyUnicode_FromString((char*)"python_function");
    // Load the module object
    pModule = PyImport_Import(pName);
    // pDict is a borrowed reference
    pDict = PyModule_GetDict(pModule);
    // pFunc is also a borrowed reference
    pFunc = PyDict_GetItemString(pDict, (char*)"function");

    if (pFunc != NULL)
    {
        if (PyCallable_Check(pFunc))
        {
            PyObject *pResult;

            pResult = PyObject_CallFunction(pFunc, "");

            Py_DECREF(pResult);
        }
        else {PyErr_Print();}
    }
    else {std::cout << "pFunc is NULL!" << std::endl;}

    // Clean up
    Py_DECREF(pFunc);
    Py_DECREF(pDict);
    Py_DECREF(pModule);
    Py_DECREF(pName);

    // Finish the Python Interpreter
    Py_Finalize();

    return 0;
}

This code compiles and works perfectly on my system, but as soon as I want to call another function in a second directory, called Folder 2, I get the error: Segmentation Fault (core dumped) . This is the code:

#include <Python.h>
#include <string>
#include <iostream>

int main()
{
    PyObject *pName, *pModule, *pDict, *pFunc;

    setenv("PYTHONDONTWRITEBYTECODE", " ", 1);

    // Initialize the Python Interpreter
    Py_Initialize();

    //CALL FUNCTION FROM FOLDER 1:
    std::wstring pathWide = L"./Folder 1";
    PySys_SetPath(pathWide.c_str());

    // Build the name object
    pName = PyUnicode_FromString((char*)"python_function");
    // Load the module object
    pModule = PyImport_Import(pName);
    // pDict is a borrowed reference
    pDict = PyModule_GetDict(pModule);
    // pFunc is also a borrowed reference
    pFunc = PyDict_GetItemString(pDict, (char*)"function");

    if (pFunc != NULL)
    {
        if (PyCallable_Check(pFunc))
        {
            PyObject *pResult;

            pResult = PyObject_CallFunction(pFunc, "");

            Py_DECREF(pResult);
        }
        else {PyErr_Print();}
    }
    else {std::cout << "pFunc is NULL!" << std::endl;}

    //CALL FUNCTION FROM FOLDER 2:
    pathWide = L"./Folder 2";
    PySys_SetPath(pathWide.c_str());

    // Build the name object
    pName = PyUnicode_FromString((char*)"python_function");
    // Load the module object
    pModule = PyImport_Import(pName);
    // pDict is a borrowed reference
    pDict = PyModule_GetDict(pModule);
    // pFunc is also a borrowed reference
    pFunc = PyDict_GetItemString(pDict, (char*)"function");

    if (pFunc != NULL)
    {
        if (PyCallable_Check(pFunc))
        {
            PyObject *pResult;

            pResult = PyObject_CallFunction(pFunc, "");

            Py_DECREF(pResult);
        }
        else {PyErr_Print();}
    }
    else {std::cout << "pFunc is NULL!" << std::endl;}

    // Clean up
    Py_DECREF(pFunc);
    Py_DECREF(pDict);
    Py_DECREF(pModule);
    Py_DECREF(pName);

    // Finish the Python Interpreter
    Py_Finalize();

    return 0;
}

The error occurs after I call the first function, so it seems like its not changing directories or something. I'm using Ubuntu and I have python 3.4

I have tried other methods of changing directories, not just PySys_SetPath , but also setenv("PYTHONPATH", path, 1);

NOTE: I'm not worried about error detection right now, I'd rather have code that works in ideal circumstances, then worry about imperfect circumstances.

EDIT:

Debug output:

#0 0x7ffff79b16cb   PyModule_GetDict() (/usr/lib/x86_64-linux-gnu/libpython3.4m.so.1.0:??)
#1 0x4010e6 main() (/home/ben/Documents/Programming/Projects/PYTHON TEST/main.cpp:23)

Oddly the debug says that the error happens at line 23, but line 23 doesn't cause an error if you run the first code segment

IN RESPONSE TO PETER BRITTAIN'S ANSWER:

If I replace the second PyImport_Import() with PyImport_ReloadModule() , I get an error printed to the console, like this:

ImportError: No module named 'imp'
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 53, in apport_excepthook
    if not enabled():
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 24, in enabled
    import re
ImportError: No module named 're'

Original exception was:
ImportError: No module named 'imp'

Answer 1

EDIT: Updated with further answers to the bugs found.

You're failing to import your module in your debug output. When run outside of the debugger, you then hit the issue that you can't just import with the same import call. The full chain of problems is something like this.

When debugging:

1. Your debugger is not setting the right current working directory.
2. The relative path is not valid but is accepted by PySys_SetPath() .
3. You therefore get NULL from PyImport_Import() indicating that the import fails (as documented here under the debugger.
4. Since you aren't checking for errors, you pass NULL into the next function, which tries to dereference the pointer and fails with a segmentation fault.

I hit an issue with Python2.7 (using char* rather than wchar* - as covered in the comments below). Putting that to one side, when running normally:

1. The relative path is valid and accepted by PySys_SetPath() .
2. You therefore manage to load the module the first time.
3. You then run through the rest of the code, but this time it segmentation faults at the second import. This is because you can't reload the module this way. You need to use PyImport_ReloadModule() instead.
4. Even with that fix, you find you can't load standard libraries. That's because you've removed the rest of your system path. You need to follow the advice here .

So, the fixes are:

1. Use std::string instead of std::wstring (for Python 2.x).
2. Use PyImport_ReloadModule() when reloading modules.
3. Make sure that you include the full sys.path when setting your path.
4. Check your errors - I got explicit module import errors using PyErr_Print() for most of the problems.