Python multiprocess: synchronization of processes not spawned by the same script

Question

As per subject, I'm trying to understand if there is a way to synchronize different processes which have not been created by a main/master script.

The scenario is this: my python script is scheduled via Windows scheduler several times per day, sometimes also in parallel. The script instances have to write information on a certain set of shared files, but since I can have concurrent access to those files I need a way to synchronize them. I'm looking to the multiprocess module in python, but it seems to me that it only handles scenarios in which the subprocesses are spawned by a main one. Am I correct? can you point me out strategies to achieve what I need?

Answer 1

I'm facing the same problem and the best solution found so far (I'm not really happy with it, actually) is FileLock.

Here's an example:

from filelock import FileLock

if __name__ == "__main__":
    lock_file_name = 'lockfile.lck'
    lock = FileLock(lock_file_name)
    with lock:
        print('Hi from Python script!')
        for i in range(1, 1000):
            print(f"Current value is: {i}")
        print('Python script ended!')

Here, even launching multiple instances of the script at the same time, the code in the with lock: block is executed in exclusive way.