I have a question quite similar to this question, where I need the follow conditions to be upheld:
The solution posted in the linked question uses a third party library which adds an arbitrary .LOCK
file in the same directory as the file in question. It is a solution that only works wrt to the program in which that library is being used and doesn't prevent any other process/program from using the file as they may not be implemented to check for a .LOCK
association.
In essence, I wish to replicate this result using only Python's standard library.
BLUF : Need a standard library implementation specific to Windows for exclusive file locking
To give an example of the problem set, assume there is:
Suppose that User 1 is running Program A on the file and at some point the following is executed:
with open(fp, 'rb') as f:
while True:
chunk = f.read(10)
if chunk:
# do something with chunk
else:
break
Thus they are iterating through the file 10 bytes at a time.
Now User 2 runs Program B on the same file a moment later:
with open(fp, 'wb') as f:
for b in data: # some byte array
f.write(b)
On Windows, the file in question is immediately truncated and Program A stops iterating (even if it wasn't done) and Program B begins to write to the file. Therefore I need a way to ensure that the file may not be opened in a different mode that would alter its content if previously opened.
I was looking at the msvcrt library, namely the msvcrt.locking()
interface. What I have been successful at doing is ensuring that a file opened for reading can be locked for reading, but nobody else can read the file (as I lock the entire file):
>>> f1 = open(fp, 'rb')
>>> f2 = open(fp, 'rb')
>>> msvcrt.locking(f1.fileno(), msvcrt.LK_LOCK, os.stat(fp).st_size)
>>> next(f1)
b"\x00\x05'\n"
>>> next(f2)
PermissionError: [Errno 13] Permission denied
This is an acceptible result, just not the most desired.
In the same scenario, User 1 runs Program A which includes:
with open(fp, 'rb') as f
msvcrt.locking(f.fileno(), msvcrt.LK_LOCK, os.stat(fp).st_size)
# repeat while block
msvcrt.locking(f.fileno(), msvcrt.LK_UNLCK, os.stat(fp).st_size)
Then User 2 runs Program B a moment later, the same result occurs and the file is truncated.
At this point, I would've liked a way to throw an error to User 2 stating the file is opened for reading somewhere else and cannot be written at this time. But if User 3 came along and opened the file for reading, then there would be no problem.
Update:
A potential solution is to change the permissions of a file (with exception catching if the file is already in use):
>>> os.chmod(fp, stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH)
>>> with open(fp, 'wb') as f:
# do something
PermissionError: [Errno 13] Permission denied <fp>
This doesn't feel like the best solution (particularly if the users didn't have the permission to even change permissions). Still looking for a proper locking solution but msvcrt
doesn't prevent truncating and writing if the file is locked for reading. There still doesn't appear to be a way to generate an exclusive lock with Python's standard library.
For those who are interested in a Windows specific solution:
import os
import ctypes
import msvcrt
import pathlib
# Windows constants for file operations
NULL = 0x00000000
CREATE_ALWAYS = 0x00000002
OPEN_EXISTING = 0x00000003
FILE_SHARE_READ = 0x00000001
FILE_ATTRIBUTE_READONLY = 0x00000001 # strictly for file reading
FILE_ATTRIBUTE_NORMAL = 0x00000080 # strictly for file writing
FILE_FLAG_SEQUENTIAL_SCAN = 0x08000000
GENERIC_READ = 0x80000000
GENERIC_WRITE = 0x40000000
_ACCESS_MASK = os.O_RDONLY | os.O_WRONLY
_ACCESS_MAP = {os.O_RDONLY: GENERIC_READ,
os.O_WRONLY: GENERIC_WRITE
}
_CREATE_MASK = os.O_CREAT | os.O_TRUNC
_CREATE_MAP = {NULL: OPEN_EXISTING,
os.O_CREAT | os.O_TRUNC: CREATE_ALWAYS
}
win32 = ctypes.WinDLL('kernel32.dll', use_last_error=True)
win32.CreateFileW.restype = ctypes.c_void_p
INVALID_FILE_HANDLE = ctypes.c_void_p(-1).value
def _opener(path: pathlib.Path, flags: int) -> int:
access_flags = _ACCESS_MAP[flags & _ACCESS_MASK]
create_flags = _CREATE_MAP[flags & _CREATE_MASK]
if flags & os.O_WRONLY:
share_flags = NULL
attr_flags = FILE_ATTRIBUTE_NORMAL
else:
share_flags = FILE_SHARE_READ
attr_flags = FILE_ATTRIBUTE_READONLY
attr_flags |= FILE_FLAG_SEQUENTIAL_SCAN
h = win32.CreateFileW(path, access_flags, share_flags, NULL, create_flags, attr_flags, NULL)
if h == INVALID_FILE_HANDLE:
raise ctypes.WinError(ctypes.get_last_error())
return msvcrt.open_osfhandle(h, flags)
class _FileControlAccessor(pathlib._NormalAccessor):
open = staticmethod(_opener)
_control_accessor = _FileControlAccessor()
class Path(pathlib.WindowsPath):
def _init(self) -> None:
self._closed = False
self._accessor = _control_accessor
def _opener(self, name, flags) -> int:
return self._accessor.open(name, flags)
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.