python tempfile和多处理池错误

Question

I'm experimenting with python's multiprocessing. I struggled with a bug in my code and managed to narrow it down. However, I still don't know why this happens. What I'm posting is just sample code. If I import tempfile module and change tempdir, the code crashes at pool creation. I'm using python 2.7.5

Here's the code

from multiprocessing import Pool
import tempfile

tempfile.tempdir = "R:/" #REMOVING THIS LINE FIXES THE ERROR

def  f(x):
 return x*x

if __name__ == '__main__':
 pool = Pool(processes=4)              # start 4 worker processes
 result = pool.apply_async(f, [10])    # evaluate "f(10)" asynchronously
 print result.get(timeout=1)           # prints "100" unless your computer is *very* slow
 print pool.map(f, range(10))          # prints "[0, 1, 4,..., 81]"

Here's error

R:\>mp_pool_test.py
Traceback (most recent call last):
  File "R:\mp_pool_test.py", line 11, in <module>
pool = Pool(processes=4)              # start 4 worker processes
File "C:\Python27\lib\multiprocessing\__init__.py", line 232, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild)
File "C:\Python27\lib\multiprocessing\pool.py", line 138, in __init__
self._setup_queues()
File "C:\Python27\lib\multiprocessing\pool.py", line 233, in _setup_queues
self._inqueue = SimpleQueue()
File "C:\Python27\lib\multiprocessing\queues.py", line 351, in __init__
self._reader, self._writer = Pipe(duplex=False)
File "C:\Python27\lib\multiprocessing\__init__.py", line 107, in Pipe
return Pipe(duplex)
File "C:\Python27\lib\multiprocessing\connection.py", line 223, in Pipe
1, obsize, ibsize, win32.NMPWAIT_WAIT_FOREVER, win32.NULL
WindowsError: [Error 123] The filename, directory name, or volume label syntax is incorrect

This code works fine.

from multiprocessing import Pool
import tempfile as TF

TF.tempdir = "R:/"

def  f(x):
    return x*x

if __name__ == '__main__':
 print("test")

The bizarre thing is that, both times I don't do anything with TF.tempdir, but the one with the Pool doesn't work for some reason.

Answer 1

It is cool it looks like you have a name collision from what I can see in

"C:\\Program Files\\PYTHON\\Lib\\multiprocessing\\connection.py"

It seems that multipprocessing is using tempfile as well

That behavior should not happen but it looks to me like the problem is in line 66 of connection.py

elif family == 'AF_PIPE':
    return tempfile.mktemp(prefix=r'\\.\pipe\pyc-%d-%d-' %
                           (os.getpid(), _mmap_counter.next()))

I am still poking at this, I looked at globals after importing tempfile and then tempfile as TF, different names exist but now I am wondering about references, and so am trying to figure out if they point to the same thing.