Threading and target function in external file (python)

Question

I want to move some functions to an external file for making it clearer. lets say i have this example code (which does indeed work):

import threading
from time import sleep

testVal = 0

def testFunc():
   while True:
        global testVal
        sleep(1)
        testVal = testVal + 1
        print(testVal)

t = threading.Thread(target=testFunc, args=())
t.daemon = True
t.start()

try:
    while True:
        sleep(2)
        print('testval = ' + str(testVal))
except KeyboardInterrupt:
    pass

now i want to move testFunc() to a new python file. My guess was the following but the global variables don't seem to be the same.

testserver.py:

import threading
import testclient
from time import sleep

testVal = 0

t = threading.Thread(target=testclient.testFunc, args=())
t.daemon = True
t.start()

try:
    while True:
        sleep(2)
        print('testval = ' + str(testVal))
except KeyboardInterrupt:
    pass

and testclient.py:

from time import sleep
from testserver import testVal as val

def testFunc():
    while True:
        global val
        sleep(1)
        val = val + 1
        print(val)

my output is:

1 testval = 0 2 3 testval = 0 (testval didn't change) ...

while it should:

1 testval = 1 2 3 testval = 3 ...

any suggestions? Thanks!

Answer 1

Your immediate problem is not due to multithreading (we'll get to that) but due to how you use global variables. The thing is, when you use this:

from testserver import testVal as val

You're essentially doing this:

import testserver
val = testserver.testVal

ie you're creating a local reference val that points to the testserver.testVal value. This is all fine and dandy when you read it (the first time at least) but when you try to assign its value in your function with:

val = val + 1

You're actually re-assigning the local (to testclient.py) val variable, not setting the value of testserver.testVal . You have to directly reference the actual pointer (ie testserver.testVal += 1 ) if you want to change its value.

That being said, the next problem you might encounter might stem directly from multithreading - you can encounter a race-condition oddity where GIL pauses one thread right after reading the value, but before actually writing it, and the next thread reading it and overwriting the current value, then the first thread resumes and writes the same value resulting in single increase despite two calls. You need to use some sort of mutex to make sure that all non-atomic operations execute exclusively to one thread if you want to use your data this way. The easiest way to do it is with a Lock that comes with the threading module:

testserver.py:

# ...
testVal = 0
testValLock = threading.Lock()
# ...

testclient.py:

# ...
with testserver.testValLock:
    testserver.testVal += 1
# ...

A third and final problem you might encounter is a circular dependency (testserver.py requires testclient.py, which requires testserver.py) and I'd advise you to re-think the way you want to approach this problem. If all you want is a common global store - create it separately from modules that might depend on it. That way you ensure proper loading and initializing order without the danger of unresolveable circular dependencies.