[英]Python -> Copy directory while another process removes/adds files from/to it
I work with Windows 10 with Python3.6. 我使用带有Python3.6的Windows 10。
There are two processes: 有两个过程:
My process (COPY): Copy a directory X (get as much as possible). 我的过程(COPY):复制目录X(获取尽可能多的目录)。 (i do it via shutil.copytree)
(我通过shutil.copytree做到)
Foreign process (CREATOR): Deletes and Recreates files in directory X occasionally. 外部进程(CREATOR):偶尔删除和重新创建目录X中的文件。
Problem: One of each process 'crashes' occasionally when both processes try to operate on the directory X on the same time. 问题:当两个进程试图同时在目录X上运行时,每个进程之一偶尔会崩溃。 I can handle exceptions in my process but do not want the foreign process to crash.
我可以在流程中处理异常,但不希望外部流程崩溃。
CREATOR: PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: Filepath 创建者:PermissionError:[WinError 32]该进程无法访问文件,因为该文件正被另一个进程使用:Filepath
I can only change code for my process. 我只能为自己的过程更改代码。 Is there a way to copy files (files do not have to be complete) without disturb foreign process?
有没有一种方法可以复制文件(文件不一定是完整的)而不会干扰外部进程?
Background: Multiple foreign processes create test reports containing many html files. 背景:多个外部流程会创建包含许多html文件的测试报告。 I want to copy these to one location during test execution and be able to view the current result from there.
我想在测试执行期间将它们复制到一个位置,并能够从那里查看当前结果。
import shutil, os, stat
import multiprocessing as MP
from time import sleep
import random
from traceback import format_exc
work_dir = r"C:\\tmp\\Report"
dest = r"C:\\tmp\\Report_Copy"
def CREATOR():
while(True):
if not(os.path.exists(work_dir)):
os.mkdir(work_dir)
print("CREATOR: Created %s" % work_dir)
for root, dirs, files in os.walk(work_dir, topdown=False):
# remove current files
for name in files:
fullName = os.path.join(root, name)
os.chmod(fullName, stat.S_IWRITE)
os.remove(fullName)
print("CREATOR: Removed %s" % work_dir)
for i in range(1000):
tmpPath = os.path.join(work_dir, "result%d" % i)
with open(tmpPath, "w") as f:
f.write("voodoo"*50000)
print("CREATOR: Created 100 files")
sleep(0)
def COPY():
while(True):
try:
sleep(1)
# SOLUTION FOR WINDOWS
#######
os.system("robocopy %s %s /E >nul 2>&1" %(work_dir, dest))
#######
#shutil.copytree(work_dir, dest)
print("COPY: copied %s to %s" % (work_dir, dest))
shutil.rmtree(dest)
print("COPY: removed %s" % dest)
except:
print("COPY:\n" + format_exc())
shutil.rmtree(dest)
if __name__ == "__main__":
if os.path.exists(work_dir):
shutil.rmtree(work_dir)
if os.path.exists(dest):
shutil.rmtree(dest)
P1 = MP.Process(target=CREATOR)
P1.start()
P2 = MP.Process(target=COPY)
P2.start()
P1.join()
P2.join()
COPY process can fail occasionally but CREATOR process should never crash because COPY process is accessing files. COPY进程有时可能会失败,但是CREATOR进程永远不会崩溃,因为COPY进程正在访问文件。 The code is only an example.
该代码仅是示例。 I can not change the code from CREATOR (foreign) process!
我无法从CREATOR(外部)过程更改代码!
This was solved by me by using a OS specific copy tool under Windows. 我通过使用Windows下特定于操作系统的复制工具解决了这一问题。
I used robocopy in COPY Process instead of shutil.copytree
( shutil
worked under Linux though). 我在COPY进程中使用了robocopy而不是
shutil.copytree
( shutil
在Linux下工作)。
os.system("robocopy %s %s /E >nul 2>&1" %(src, dest))
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.