Django Celery: Model object does not exists within the celery's task (ATOMIC_REQUESTS=False)

Question

I am getting MyModel matching query does not exist. error while fetch the object which I am creating before entering the celery task. I am calling the task from within my APIView .

my_model_obj = MyModel(x=1, y=2)
my_model_obj.save()
my_celery_task.delay(my_model_obj.id)

within my task function, I am doing:

@task()
def my_celery_task(my_model_id):
    MyModel.objects.get(id=my_model_id)

I do not have ATOMIC_REQUESTS param in my Django's DATABASE configuration. So, by default it should be False.

I believe it is because the Django is releasing control from the model object even before the data is been actually saved to the DB. This is an intermittent issue which happens sometime and sometime it works fine.

Earlier I had the similar issue in which I was updating the values of model object but the updated values were not reflecting within the celery's task. In order to make that run, I have added 10 secs of delay. But this time I am looking for some permanent solution. Is there some way to fix this? I haven't got any configuration param in neither Django's or Celery's configuration to deal with this kind of behavior.

Answer 1

The issue was because I was using TransactionMiddleware , and it does similar thing as @transaction.commit_on_success decorator. If you want to keep using TransactionMiddleware , you should look into using @transaction.autocommit decorator in your views with celery tasks, or @transaction.commit_manually