celery task is not called when it is shown on flower dashboard

Question

I am using celery alongside fastapi. I could see celery task registered from flower dashboard, however, the task does not get called. I could neither see any error or any kind of logs.

This is how I have setup

app/app/core/celery_app.py

from celery import Celery

celery_app = Celery("workers", broker="amqp://guest@queue//")

celery_app.conf.task_routes = {
    "app.workers.download_and_generate_logs_df.download_and_generate_logs_df": "main-queue"
}

app/app/workers/download_and_generate_logs_df.py

from app.core.celery_app import celery_app


@celery_app.task()
def download_and_generate_logs_df(repo, token) -> str:
    print("celery task entered")
    if repo.provider == "gitlab":
        gitlab_repo_url = f"https://oauth2:{token}@gitlab.com/{repo.fullpath}"
        root_dir = os.getcwd()  # /app
        clone_url = f"git clone {gitlab_repo_url} --bare"
        os.chdir("./app/bare-gits/")
        gits_dir = os.getcwd()
        if os.path.isdir(gits_dir):
            if os.path.isdir(f"{repo.name}.git"):
                # create a logs file and save it to logs folder
                print("already exist")
                generate_logs(repo.name)
                return {"msg": "Calculated"}
        else:
            os.system(clone_url)
            generate_logs(repo.name)
            return {"msg": "Cloned and calculated"}

        os.chdir(root_dir)

app/app/worker-start.sh

celery -A app.core.celery_app worker -l info -Q main-queue -c 1

app/app/main.py

@app.get("/test-repo-calculation")
def test_repo_calculation(*, db: Session = Depends(get_db), user_id: int, repo_id: int):
    celery_task = celery_app.send_task(
                "app.workers.download_and_generate_logs_df",
                args=[repo.name, token],
            )
    print("celery_task", celery_task)
    return {"msg": "Calculation is in progress"}

This is what I see in flower dashboard and logs

This is what my project structure looks like

Answer 1

Not sure if it's related but according to docs :

celery_app.conf.task_routes = {
    "app.workers.download_and_generate_logs_df.download_and_generate_logs_df": "main-queue"
}

should be:

celery_app.conf.task_routes = {
    "app.workers.download_and_generate_logs_df.download_and_generate_logs_df": {"queue": "main-queue"}
}

EDIT :

1. you need to register your task:

celery_app = Celery("workers", broker="amqp://guest@queue//")

should be:

celery_app = Celery("workers", broker="amqp://guest@queue//", include=["app.workers.download_and_generate_logs_df"])

2. change:

celery_task = celery_app.send_task(
                "app.workers.download_and_generate_logs_df",
                args=[repo.name, token],
            )

to:

celery_task = celery_app.send_task(
                "app.workers.download_and_generate_logs_df.download_and_generate_logs_df",
                args=[repo.name, token],
            )