Python module code says path doesn't exist but is readable and statable

Question

I'm running into a really bizarre problem in a python module and I'm not sure whether or not it's due to some python weirdness or some brokenness in windows or the filesystem being in a weird state.

Here's the problem: a module I'm using creates some output that goes into a specified directory, and it creates that directory if it doesn't exist. This works fine if you run it once, running it a second time the module thinks that the directory doesn't exist, tries to create it, then bombs out with a "Cannot create a file when that file already exists" error.

So, I went digging. In my main script I can run os.path.exists on the directory and it says it exists just fine. In the module the same exact code returns false however. Also in the module if I do an os.access( path , os.R_OK) it returns true, indicating the path is readable, and if I do an os.stat( path ) it returns file info, indicating the path exists.

If I run the same code manually in a python shell it all works fine, says the path exists, etc.

Here's some code and the outputs:

In my main script:

print('Path {} exists: {} - readable: {} - stat: {}', 
    filename, 
    os.path.exists(filename), 
    os.access(filename, os.R_OK), 
    os.stat(filename))

Output:

File testoutput exists: True - readable: True - stat: nt.stat_result(st_mode=16895, st_ino=0L, st_dev=0L, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1508458382L, st_mtime=1508458382L, st_ctime=1508458382L)

Inside the module, the same code:

File testoutput exists: False - readable: True - stat: nt.stat_result(st_mode=16895, st_ino=0L, st_dev=0L, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1508458382L, st_mtime=1508458382L, st_ctime=1508458382L)

What the heck is going on here? Is the module code running in some sort of different context with different permissions? Is my filesystem broken in some weird way?

Also, I should mention that all of this is being done in an elevated cmd shell so that's not the problem.

---

Solved:

It turned out to be some intervening code that was monkey patching os.path.exists, the most helpful tool was using reflection and the "inspect" module to figure out where the code was actually coming from, specifically:

    print('os.path: {} - {} - {} - {}'.format(os.__file__, os.path.__name__, os.path.__file__, os.path.exists.__module__))
    print(inspect.getsource(os.path.exists))

The __module__ info told me where the monkeypatching was happening.

Answer 1

This is just an educated guess. os.path.exists will return False for a broken symlink, while os.access will report the file itself (the symlink) as being readable. Is it possible your file is a symlink?

Another difference is that os.access will use the real (not effective) uid/gid to test readability while os.path.exists uses the effective uid/gid. Is it possible there is a change of user ID between the modules?