Launching a Python watchdog in an Azure Devops Release pipeline

Question

I have a situation where I would like to start up two Python watchdogs consecutively in an Azure Release stage and then continue with the following tasks.

As I understand, the subprocess.Popen is the way to go if you want to create a "fire & forget" behavior like this When I run something like subprocess.Popen(["python", "mywatchdog1.py"]) outside of Azure it behaves as I expect, it "fires & forgets" , but when running that same call from a task (tried with both "Powershell" and "Run a Python Script") in Azure Devops, the task stops and waits for the watchdog process to finish.

Here's an example:

# mywatchdog1.py

import time
import os
from watchdog.observers import Observer
from watchdog.events import LoggingEventHandler

class MyWatchdogEventHandler(LoggingEventHandler):

    def dispatch(self, event):
        print(f"A new dog was created")


if __name__ == "__main__":
    dog_file_path = "c:\\dogs"
    event_handler = MyWatchdogEventHandler()
    observer = Observer()
    observer.schedule(event_handler, dog_file_path, recursive=True)
    observer.start()
    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

This is what I'm trying to launch using something like:

import subprocess as sp

if __name__ == "__main__":
    sp.Popen(["python", "C:\\scripts\\mywatchdog01.py"])

So, how do I make subprocess.Popen behave the same way regardless if you run locally or on an Azure Windows 2016 agent? Any help would be greatly appreciated.

/Niklas

Answer 1

So, how do I make subprocess.Popen behave the same way regardless if you run locally or on an Azure Windows 2016 agent?

Sorry but I'm afraid this is not supported scenario for now. Azure Devops pipeline executes the tasks one by one, it won't finish current task until it finishes all the job to be done within it.

That's why it hangs in your Powershell or Run a Python Script task cause the watchdog is always listening to inputs. You can refer to this document for more details. And it's by design that the task won't finish if there's something like watchdog within the task keeping monitoring the file explorer changes.

Hope my answer helps to resolve your puzzle :)