Python How to run multiple functions every n seconds without interference

Question

So assume I'm trying to automate tasks on a webpage with selenium.

I have multiple buttons I have to click, which take me to different parts of the page where I have to run different functions to do different tasks. With each click of these buttons, when the corresponding tasks have been completed, a timer appears in the place of the button. When that timer runs out, I have to click that button again and run the corresponding function of tasks, then rinse and repeat.

Here is the code I've tried:

import selenium
import threading

driver = webdriver.Chrome()
driver.get("https://example.com")

def foo1():
    button1 = driver.find_element_by_id("button1")
    button1.click()
    # do stuff
    timer = int(button1.text)
    threading.Timer(timer, foo1).start()

def foo2():
    button2 = driver.find_element_by_id("button2")
    button2.click()
    # do stuff
    timer = int(button2.text)
    threading.Timer(timer, foo2).start()

def foo3():
    button3 = driver.find_element_by_id("button3")
    button3.click()
    # do stuff
    timer = int(button3.text)
    threading.Timer(timer, foo3).start()

foo1()
foo2()
foo3()

Now, my problem with this is that, because there are multiple of these buttons, the scenario of one of the timers running out while another function is running can occur. This causes the script to click away without finishing the current sequence of tasks, thus terminating one of the threads.

I've tried using threading.Lock() objects but they didn't fix the issue. Do I have the wrong approach or am I just missing something here?

Answer 1

The least painful solution to your problem is by using multiple webdriver objects, each of the objects in a different thread, and use a dictionary of the variables you want to be communicated.

import time
def foo():
    driver = webdriver.Chrome()
    driver.get("https://example.com")
    button = driver.find_element_by_id("button")
    while True:
        button.click()
        # do stuff
        timer = int(button.text)
        time.sleep(timer)

As far as I know the webdriver object is not thread-safe, so it is better to replicate it across mulitple threads , than using the same object over and over again in different threads. Also in your solution, you spawn different threads in every different loop iteration, which can have many repercussions. You can view this post with a similar situation in C++

Answer 2

From what I know, the thread initialization can be changed to solve the problem.

import selenium
import threading
import time 

driver = webdriver.Chrome()
driver.get("https://example.com")

def foo1():
    button1 = driver.find_element_by_id("button1")
    button1.click()
    # do stuff
    timer = int(button1.text)
    time.sleep(timer)
    foo1()
    
def foo2():
    button2 = driver.find_element_by_id("button2")
    button2.click()
    # do stuff
    timer = int(button2.text)
    time.sleep(timer)
    foo2()

t1 = threading.Thread(target=foo1)
t2 = threading.Thread(target=foo2)
t1.start()
t2.start()

Answer 3

You can keep the same structure of code by using a Lock object. However, you only have to define one Lock object and acquire/release it in every function, like this:

import selenium
import threading

driver = webdriver.Chrome()
driver.get("https://example.com")
lock = threading.Lock()

def foo1():
    lock.acquire()
    button1 = driver.find_element_by_id("button1")
    button1.click()
    # do stuff
    timer = int(button1.text)
    lock.release()
    threading.Timer(timer, foo1).start()

def foo2():
    lock.acquire()
    button2 = driver.find_element_by_id("button2")
    button2.click()
    # do stuff
    timer = int(button2.text)
    lock.release()
    threading.Timer(timer, foo2).start()

def foo3():
    lock.acquire()
    button3 = driver.find_element_by_id("button3")
    button3.click()
    # do stuff
    timer = int(button3.text)
    lock.release()
    threading.Timer(timer, foo3).start()

foo1()
foo2()
foo3()

By doing so, you ensure that your function is atomic. For instance, if foo2 is called while foo1 is executing, then foo2 will be executed once foo1 calls lock.release() .