Celery 'module' object has no attribute 'app' when using Python 3

Question

I am going through Celery tutorial. They are using Python2 and I am trying to implement the same using python3.

I have 2 files:

celery_proj.py :

from celery import Celery

app = Celery(
    'proj', broker='amqp://', backend='amqp://', include=['proj.tasks'])

app.conf.update(Celery_TAST_RESULT_EXPIRES=3600,)

if __name__ == '__main__':
    app.start()

and tasks.py :

from celery_proj import app


@app.task
def add(x, y):
    return x + y


@app.task
def mul(x, y):
    return x * y


@app.task
def xsum(numbers):
    return sum(numbers)

When I try to run celery -A proj worker -l info I am getting :

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/celery/app/utils.py", line 235, in find_app
    found = sym.app
AttributeError: 'module' object has no attribute 'app'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/celery", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.4/dist-packages/celery/__main__.py", line 30, in main
    main()
  File "/usr/local/lib/python3.4/dist-packages/celery/bin/celery.py", line 81, in main
    cmd.execute_from_commandline(argv)
  File "/usr/local/lib/python3.4/dist-packages/celery/bin/celery.py", line 769, in execute_from_commandline
    super(CeleryCommand, self).execute_from_commandline(argv)))
  File "/usr/local/lib/python3.4/dist-packages/celery/bin/base.py", line 309, in execute_from_commandline
    argv = self.setup_app_from_commandline(argv)
  File "/usr/local/lib/python3.4/dist-packages/celery/bin/base.py", line 469, in setup_app_from_commandline
    self.app = self.find_app(app)
  File "/usr/local/lib/python3.4/dist-packages/celery/bin/base.py", line 489, in find_app
    return find_app(app, symbol_by_name=self.symbol_by_name)
  File "/usr/local/lib/python3.4/dist-packages/celery/app/utils.py", line 240, in find_app
    found = sym.celery
AttributeError: 'module' object has no attribute 'celery'

What am I doing wrong and how to fix this?

Answer 1

If you're using Python 3, you can use absolute imports via: from __future__ import absolute_import

Here's a sample celery.py from a recent Python 3, Django 1.7 app of mine:

from __future__ import absolute_import

import os
import django

from celery import Celery
from django.conf import settings


# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'my_app.settings')
django.setup()

app = Celery('my_app')
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

Answer 2

When you run Celery with celery -A proj worker

AttributeError: 'module' object has no attribute 'app'
...
AttributeError: 'module' object has no attribute 'celery'

tell you that it tries to find:

a) app instance

b) proj.celery module

So you can either add from celery_proj import app to __init__.py or rename your celery_proj.py to celery.py .

Or you can run Celery as celery -A proj.celery_proj worker

I found the answer here as no answer here on stackoverflow had helped me.

Answer 3

I've been playing around and have understood one interesting thing. When we run celery -A proj worker -l info we basically run proj folder. I believe that, when you run proj folder, celery looks for celery.py file there. It was OK in python2, as we used absolute imports and could write from proj.celery import ... but in python3 here is no such possibility. We need to write from celery import ... and this will cause an error as celery is a module, so we need to rename celery.py to something else. When we do this we can't run proj any more. Maybe I am wrong, but at least I made it work...

What you need to do is open proj directory and run tasks.py from there, only then will you be able to use from celery_proj import app and keep celery_proj .

Please write if I am mistaken and add your solutions.

Answer 4

I had the same problem when running the celery and django app in a docker container. Looks like the celery worker command searches the current directory for the celery app. When the command was run from any arbitrary directory, this attribute not found error occurs. But when it is run from the django app directory (where it can find the app/modules), the celery worker runs as expected.

Answer 5

I came across a strange case with docker when i provided the same name for both main project directory and app directory.

Correcting the path in the volume solved issue to me( - ./app:/app) check the celery configration part in the docker section.

-app:
    ..
    app:
        __init__.py
        celery.py

init .py

from __future__ import absolute_import

from .celery import app as celery_app

__all__ = ['celery_app']

celery.py

from __future__ import absolute_import, unicode_literals
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'app.settings')

app = Celery('app')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

Hence i had to provide proper path in the volume section

celery:
    build: .
    command: celery -A app worker -l info
    volumes:
     - ./app:/app
    depends_on:
     - db
     - redis