[英]Prevent a file from being created in python
I'm working on a python server which concurrently handles transactions on a number of databases, each storing performance data about a different application. 我正在使用python服务器,该服务器可同时处理多个数据库上的事务,每个数据库都存储有关不同应用程序的性能数据。 Concurrency is accomplished via the Multiprocessing module, so each transaction thread starts in a new process, and shared-memory data protection schemes are not viable.
并发是通过多处理模块完成的,因此每个事务线程都在一个新进程中启动,并且共享内存数据保护方案不可行。 I am using sqlite as my DBMS, and have opted to set up each application's DB in its own file.
我使用sqlite作为我的DBMS,并且选择在其自己的文件中设置每个应用程序的数据库。 Unfortunately, this introduces a race condition on DB creation;
不幸的是,这为数据库创建引入了竞争条件。 If two process attempt to create a DB for the same new application at the same time, both will create the file where the DB is to be stored.
如果两个进程试图同时为相同的新应用程序创建数据库,则两个进程都会创建要在其中存储数据库的文件。 My research leads me to believe that one cannot lock a file before it is created;
我的研究使我相信,在创建文件之前,无法锁定文件。 Is there some other mechanism I can use to ensure that the file is not created and then written to concurrently?
我是否可以使用其他机制来确保未创建文件然后将其并发写入?
Thanks in advance, David 预先感谢大卫
The usual Unix-style way of handling this for regular files is to just try to create the file and see if it fails. 处理常规文件的通常的Unix风格方法是尝试创建文件并查看其是否失败。 In Python's case, that would be:
在Python的情况下,将是:
try:
os.open(filename, os.O_WRONLY | os.O_CREAT | os.O_EXCL)
except IOError: # or OSError?
# Someone else created it already.
At the very least, you can use this method to try to create a "lock file" with a similar name to the database. 至少,您可以使用此方法尝试创建一个与数据库名称相似的“锁定文件”。 If the lock file is created, you go ahead and make the database.
如果创建了锁定文件,则继续创建数据库。 If not, you do whatever you need to for the "database exists" case.
如果没有,那么您需要为“数据库存在”情况做任何事情。
Name your database files in such a way that they are guaranteed not to collide. 以确保它们不会冲突的方式命名数据库文件。
http://docs.python.org/library/tempfile.html http://docs.python.org/library/tempfile.html
尝试在代码中和异常处理程序中创建文件时,可以捕获错误,检查文件是否存在并使用现有文件而不是创建文件。
You didn't mention the platform, but on linux open()
, or os.open()
in python, takes a flags parameter which you can use. 您没有提到平台,但是在linux
open()
或python中的os.open()
上,使用了可以使用的flags参数。 The O_CREAT
flag creates a file if it does not exist, and the O_EXCL
flag gives you an error if the file already exists. 如果文件不存在,
O_CREAT
标志将创建一个文件,如果该文件已存在,则O_EXCL
标志会给您一个错误。 You'll also be needing O_RDONLY
, O_WRONLY
or O_RDWR
for specifying the access mode. 您还需要
O_RDONLY
, O_WRONLY
或O_RDWR
来指定访问模式。 You can find these constants in the os
module. 您可以在
os
模块中找到这些常量。
For example: fd = os.open(filename, os.O_RDWR | os.O_CREAT | os.O_EXCL)
例如:
fd = os.open(filename, os.O_RDWR | os.O_CREAT | os.O_EXCL)
You can use the POSIX O_EXCL
and O_CREAT
flags to open(2)
to guarantee that only a single process gets the file and thus the database; 您可以使用POSIX
O_EXCL
和O_CREAT
标志来open(2)
以确保只有单个进程可以获取文件,从而可以获取数据库; O_EXCL
won't work over NFSv2 or earlier, and it'd be pretty shaky to rely on it for other network filesystems. O_EXCL
不能在NFSv2或更早的版本上运行,而将其用于其他网络文件系统会非常不稳定。
The liblockfile
library implements a network-filesystem safe locking mechanism described in the open(2)
manpage, which would be convenient; liblockfile
库实现了open(2)
联机帮助页中描述的网络文件系统安全锁定机制,这很方便; but I only see pre-made Ruby and Perl bindings. 但是我只看到预制的Ruby和Perl绑定。 Depending upon your needs, maybe providing Python bindings would be useful, or perhaps just re-implementing the algorithm:
根据您的需求,也许提供Python绑定会很有用,或者只是重新实现算法:
O_EXCL Ensure that this call creates the file: if this flag is
specified in conjunction with O_CREAT, and pathname
already exists, then open() will fail. The behavior of
O_EXCL is undefined if O_CREAT is not specified.
When these two flags are specified, symbolic links are not
followed: if pathname is a symbolic link, then open()
fails regardless of where the symbolic link points to.
O_EXCL is only supported on NFS when using NFSv3 or later
on kernel 2.6 or later. In environments where NFS O_EXCL
support is not provided, programs that rely on it for
performing locking tasks will contain a race condition.
Portable programs that want to perform atomic file locking
using a lockfile, and need to avoid reliance on NFS
support for O_EXCL, can create a unique file on the same
file system (e.g., incorporating hostname and PID), and
use link(2) to make a link to the lockfile. If link(2)
returns 0, the lock is successful. Otherwise, use stat(2)
on the unique file to check if its link count has
increased to 2, in which case the lock is also successful.
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.