How to set up a timeout for multithreading?

Question

I have a function that parses documents in a MongoDB collection. I would like to set up a timeout for each thread because I won't know if function2 will take too much time to finish. I tried setting up a @timeout_decorator.timeout(60, use_signals=False) , but apparently it does not work.

Better alternatives are considered as well. Any suggestion is greatly appreciated!

def function(collection):
    with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
        for document in collection.find({}, no_cursor_timeout=True):
            executor.submit(function2, document)

def function2(collection, document):
    try: 
        ... something ...

Answer 1

Since you are using executor.submit all your submitted tasks are returning a Future object which has its own way to declare a timeout.

def function(collection):
    futures = []
    with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
        for document in collection.find({}, no_cursor_timeout=True):
            future = executor.submit(function2, document)
            futures.append(future)

futures = function(collection)
for future in concurrent.futures.as_completed(futures):
    result = future.result(timeout=60)

You can find more documentation here .