Django + Celery + Redis on a K8s cluster

Question

I have a Django application deployed on a K8s cluster. I need to send some emails (some are scheduled, others should be sent asynchronously), and the idea was to delegate those emails to Celery.

So I set up a Redis server (with Sentinel) on the cluster, and deployed an instance for a Celery worker and another for Celery beat.

The k8s object used to deploy the Celery worker is pretty similar to the one used for the Django application. The main difference is the command introduced on the celery worker: ['celery', '-A', 'saleor', 'worker', '-l', 'INFO']

Scheduled emails are sent with no problem (celery worker and celery beat don't have any problems connecting to the Redis server). However, the asynchronous emails - "delegated" by the Django application - are not sent because it is not possible to connect to the Redis server ( ERROR celery.backends.redis Connection to Redis lost: Retry (1/20) in 1.00 second. [PID:7:uWSGIWorker1Core0] )

Error 1:

socket.gaierror: [Errno -5] No address associated with hostname 

Error 2:

redis.exceptions.ConnectionError: Error -5 connecting to redis:6379. No address associated with hostname. 

The Redis server, Celery worker, and Celery beat are in a "redis" namespace, while the other things, including the Django app, are in the "development" namespace.

Here are the variables that I define:

- name: CELERY_PASSWORD
  valueFrom:
    secretKeyRef:
      name: redis-password
      key: redis_password
- name: CELERY_BROKER_URL
  value: redis://:$(CELERY_PASSWORD)@redis:6379/1
- name: CELERY_RESULT_BACKEND
  value: redis://:$(CELERY_PASSWORD)@redis:6379/1

I also tried to define CELERY_BACKEND_URL (with the same value as CELERY_RESULT_BACKEND ), but it made no difference.

What could be the cause for not connecting to the Redis server? Am I missing any variables? Could it be because pods are in a different namespace?

Thanks!

Answer 1

This is a community wiki answer posted for better visibility. Feel free to expand it.

Solution from @sofia that helped to fix this issue:

need to use the same namespace for the Redis server and for the Django application. In this particular case, change the namespace "redis" to "development" where the application is deployed.