[英]Python “FileExists” error when making directory
I have several threads running in parallel from Python on a cluster system.我有几个线程在集群系统上从 Python 并行运行。 Each python thread outputs to a directory
mydir
.每个 python 线程输出到目录
mydir
。 Each script, before outputting checks if mydir exists and if not creates it:每个脚本,在输出之前检查mydir 是否存在,如果不存在则创建它:
if not os.path.isdir(mydir):
os.makedirs(mydir)
but this yields the error:但这会产生错误:
os.makedirs(self.log_dir)
File "/usr/lib/python2.6/os.py", line 157, in makedirs
mkdir(name,mode)
OSError: [Errno 17] File exists
I suspect it might be due to a race condition, where one job creates the dir before the other gets to it.我怀疑这可能是由于竞争条件,其中一项工作在另一个工作之前创建目录。 Is this possible?
这可能吗? If so, how can this error be avoided?
如果是这样,如何避免这个错误?
I'm not sure it's a race condition so was wondering if other issues in Python can cause this odd error.我不确定这是一个竞争条件,所以想知道 Python 中的其他问题是否会导致这个奇怪的错误。
从 Python >=3.2
, os.makedirs()
可以采用第三个可选参数exist_ok
:
os.makedirs(mydir, exist_ok=True)
Any time code can execute between when you check something and when you act on it, you will have a race condition.在您检查某事和对某事采取行动之间的任何时间代码都可以执行时,您将遇到竞争条件。 One way to avoid this (and the usual way in Python) is to just try and then handle the exception
避免这种情况的一种方法(以及 Python 中的常用方法)是尝试然后处理异常
while True:
mydir = next_dir_name()
try:
os.makedirs(mydir)
break
except OSError, e:
if e.errno != errno.EEXIST:
raise
# time.sleep might help here
pass
If you have a lot of threads trying to make a predictable series of directories this will still raise a lot of exceptions, but you will get there in the end.如果你有很多线程试图创建一系列可预测的目录,这仍然会引发很多异常,但你最终会到达那里。 Better to just have one thread creating the dirs in that case
在这种情况下最好只有一个线程创建目录
Catch the exception and, if the errno is 17, ignore it.捕获异常,如果 errno 为 17,则忽略它。 That's the only thing you can do if there's a race condition between the
isdir
and makedirs
calls.如果
isdir
和makedirs
调用之间存在竞争条件,那么这是您唯一可以做的事情。
However, it could also be possible that a file with the same name exists - in that case os.path.exists
would return True
but os.path.isdir
returns false.但是,也有可能存在同名文件——在这种情况下,
os.path.exists
将返回True
但os.path.isdir
返回 false。
I had a similar issues and here is what I did我遇到了类似的问题,这就是我所做的
try:
if not os.path.exists(os.path.dirname(mydir)):
os.makedirs(os.path.dirname(mydir))
except OSError as err:
print(err)
Description: Just checking if the directory already exist throws this error message [Errno 17] File exists because we are just checking if the directory name exist or not which will return the directory name of the mydir value being passed but not if it already exist or not.描述:只是检查目录是否已经存在会抛出此错误消息[Errno 17] 文件存在,因为我们只是检查目录名称是否存在,这将返回正在传递的mydir值的目录名称,但如果它已经存在则不会返回或不是。 What is being missed is not checking if that directory already exist which can be done by checking the path with os.path.exists() and in there we passed the respective directory name.
错过的不是检查该目录是否已经存在,这可以通过使用os.path.exists()检查路径来完成,并在那里我们传递了相应的目录名称。
To ignore the dir or file exist error, you can try this:要忽略目录或文件存在错误,您可以尝试以下操作:
except OSError, e:
if e.errno != 17:
print("Error:", e)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.