python 2 - why does 'with' behave differently in embedded c code?

Question

I'm trying to use python 2.7.5 in ac/c++ DLL. This DLL is used by another application which has made debugging a challenge. After banging away at this for hours I have isolated the problem down to where a file read in a 'with' statement is throwing an exception. This I do not understand...'with' should absorb an exception if implemented correctly, right? Anyway calling the same python code from the command line has no problems at all.

My C/CPP DLL calls this...

def parsetest(filename):
     bytesin = getMD3Bytes(filename)
     return bytesin

def getMD3Bytes(filename):
     filename = 'lower.md3'
     bytes = ''
     valuetoreturn = 1
     try:
          with open(filename,'rb') as fileptr:
               if fileptr != None:
                    bytes = fileptr.read()
                    valuetoreturn = 333
     except:
          valuetoreturn = 991
     return valuetoreturn

If the DLL runs this code via...

pValue = PyObject_CallObject(pFunc, arguments);

And gets a result via...

iResult = PyInt_AsLong(pValue);

iResult has the value of 991 instead of 333 which would only happen if an exception had not occurred inside of 'with'. I know because I had the app calling the DLL pop up a message box with iResult in it.

Even more interesting to me, this works...

C:\Program Files (x86)\DeleD CE\Plugins>python
Python 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import parseMD3
>>> testval = parseMD3.parsetest('junk')
>>> print testval
333
>>> exit()

So why does the CLI python return a different result that the same code being call from PyObject_CallObject? Why does 'with' behave differently here?

Answer 1

with does not handle exceptions. It only ensures that the file is closed when an exception occurs. If an exception occurs in the open() expression itself, the with block isn't even entered; fileptr will never be bound to None either.

You are catching all exceptions, including keyboard interrupts and memory errors, so we cannot begin to tell why the code fails when running under C++ control here.

Stick to a limited set of exceptions instead, like IOError , and log the exception properly:

import logging

logger = logging.getLogger('__name__')

def getMD3Bytes(filename):
    filename = 'lower.md3'
    bytes = ''
    valuetoreturn = 1
    try:
        with open(filename,'rb') as fileptr:
            bytes = fileptr.read()
            valuetoreturn = 333
    except IOError:
        logger.exception('Failed to open file properly')
        valuetoreturn = 991
    return valuetoreturn

The default config for the logger will output to stderr , but you can configure it to log to a file instead:

logging.basicConfig(filename='/tmp/debug.log', level=logging.DEBUG)