Writing to file and running the files sometimes works, mostly only first one

Question

This code produces WindowsError most times, rarely (like often first time it is run) not.

""" Running hexlified codes from codefiles module prepared previously """

import tempfile
import subprocess
import threading
import os

import codefiles

if __name__ == '__main__':
    for ind, c in enumerate(codefiles.exes):
        fn = tempfile.mktemp() + '.exe'
        # for some reason hexlified code is sometimes odd length and one nibble
        if len(c) & 1:
            c += '0'
        c = c.decode('hex')
        with open(fn, 'wb') as f:
            f.write(c)

        threading.Thread(target=lambda:subprocess.Popen("", executable=fn)).start()

""" One time works, one time WindowsError 32
>>> 
132096 c:\docume~1\admin\locals~1\temp\tmpkhhxxo.exe
991232 c:\docume~1\admin\locals~1\temp\tmp9ow6zz.exe
>>> ================================ RESTART ================================
>>> 
132096 c:\docume~1\admin\locals~1\temp\tmp3hb0cf.exe
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Python27\lib\threading.py", line 810, in __bootstrap_inner
    self.run()
  File "C:\Python27\lib\threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "C:\Documents and Settings\Admin\My Documents\Google Drive\Python\Tools\runner.pyw", line 18, in <lambda>
    threading.Thread(target=lambda:subprocess.Popen("", executable=fn)).start()
  File "C:\Python27\lib\subprocess.py", line 710, in __init__
    errread, errwrite)
  File "C:\Python27\lib\subprocess.py", line 958, in _execute_child
    startupinfo)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process
991232 c:\docume~1\admin\locals~1\temp\tmpnkfuon.exe

>>> 
"""

Hexlification is done with this script, and it sometimes seems to produce odd number of nibbles which seems also odd.

# bootstrapper for running hexlified executables,
# hexlification can be done by running this file directly or calling function make
# running can be done by run function in runner module, which imports the code,
# runs them from temporary files

import os

modulename = 'codefiles.py'

def code_file_names(d='.', ext='.exe'):
    return [n for n in os.listdir(d)
                            if n.lower().endswith(ext)]

def make():
    codes = code_file_names(os.curdir)
    with open(modulename, 'a') as f:
        f.write('exes = (')
        hex_codes = [open(n, 'rb').read().encode('hex') for n in codes]
        assert all(len(c) & 1 == 0 for c in hex_codes)
        print len(hex_codes),map(len, hex_codes)
        hex_codes = [repr(c) for c in hex_codes]
        f.write(',\n'.join(hex_codes))
        f.write(')\n')


if __name__ == '__main__':
    import make_exe

    # prepare hexlified exes for exes  in directory if codefiles not prepared
    if modulename not in os.listdir('.'):
        try:
            os.remove(modulename)
        except:
            pass   
        make()

    # prepare script for py2_exe to execute the scripts by run from runner
    make_exe.py2_exe('runner.pyw', 'tools/dvd.ico')        

After Mr. Martelli's suggestion, I got the error disappear, but still not expected result.

I pass in new version of code the creation of exe file if it exists (saved file names in new creation routine). It launches both codes (two hexlified codes) when creating the files, but multiple copies of first code afterwards.

Answer 1

tempfile.mktemp() , besides being deprecated since many years because of its security problems, guarantees unique names only within a single run of a program, since, per https://docs.python.org/2/library/tempfile.html#tempfile.mktemp , "The module uses a global variable that tell it how to construct a temporary name". So on the second run, as the very clear error message tells you, "The process cannot access the file because it is being used by another process" (specifically, the process started by the previous run, ie you cannot re-write an .exe that's currently running in some process).

The fix is to make sure each run uses its own unique directory for temporary files. See mkdtemp at https://docs.python.org/2/library/tempfile.html#tempfile.mkdtemp . How to eventually clean up those temporary directories is a different issue since that can't be done as long as any process is running an .exe file within such a directory -- you'll probably need a separate "clean-up script" that does what it can for the purpose, run periodically (eg in Unix I'd use cron ) and a repository (eg a small sqlite DB, or even just a file) to record which of the temporary directories created in previous runs still exist and need to be cleared up (catching the exceptions seen when they can't get cleaned up yet, so as to retry in the future).