How do you return the result of a completed celery task and store the data in variables?

Question

I have two flask modules app.py and tasks.py .

I set up Celery in tasks.py to complete a selenium webdriver request (which takes about 20 seconds). My goal is to simply return the result of that request to app.py.

Running the Celery worker on another terminal, I can see in the console that the Celery task completes successfully and prints all the data I need from the selenium request. However, now I just want to return the task result to app.py.

How do I obtain the celery worker results data from tasks.py and store each result element as a variable in app.py?

app.py:

I define the marketplace and call the task function and request the indexed results:

import tasks

marketplace = 'cheddar_block_games'

# This is what I am trying to get back:
price_check = tasks.scope(marketplace[0])
image = tasks.scope(marketplace[1])

tasks.py:

celery = Celery(broker='redis://127.0.0.1:6379')


 @celery.task()
 def scope(marketplace):
     web.get(f'https://magiceden.io/marketplace/{marketplace}')
     price_check = WebDriverWait(web,30).until(EC.visibility_of_element_located((By.XPATH, "/html/body/div[2]/div[2]/div[3]/div[2]/div[2]/div[3]/div[2]/div[4]/div/div[2]/div[1]/div[2]/div/div[2]/div/div[2]/div/span/div[2]/div/span[1]"))).text
     image = WebDriverWait(web,30).until(EC.visibility_of_element_located((By.XPATH, "/html/body/div[2]/div[2]/div[3]/div[2]/div[2]/div[3]/div[2]/div[4]/div/div[2]/div[1]/div[2]/div/div[1]/div/div/img")))

     return (price_check, image)
        

Answer 1

This answer might be relevant: https://stackoverflow.com/a/30760142/9347535

app.py should call the task eg using scope.delay or scope.apply_async. You could then fetch the task result with AsyncResult.get(): https://docs.celeryq.dev/en/latest/userguide/tasks.html#result-backends

Since the task returns a tuple, you can store each variable by unpacking it: https://docs.python.org/3/tutorial/datastructures.html#tuples-and-sequences

The result would be something like this:

import tasks

marketplace = 'cheddar_block_games'

result = tasks.scope.delay(marketplace)
price_check, image = result.get()