How to save and delete data automatically in Django database?

Question

I am trying to build a website using Django, but I would like to update my database for every 15 mins with new scraped data. Also, I would like to delete the old data that has been saved for longer than one week.

I was able to do it manually by using Django manage.py shell , but how can I make it automatically?

Is there any library for automatic database management that is compatible with Django?

Thank you in advance!

Answer 1

Having just implemented something similar myself, I'd have to agree with the comments in saying that this is a relatively involved process. Nevertheless, it is quite doable, and there is a ton of info out there on the particulars. I'll give you an overview of what I did.

My goal was to automatically run a function in my Django app every weekday at 16:00.

A quick search will tell you that you want to set up a periodic task schedule. For that I needed:

• Celery - A popular task queue/scheduling system with Django compatibility
• Redis - The broker I opted to use
• Django Celery Beat - Celery Beat is the periodic task scheduling part of Celery - this Django package adds database integrations so that you can control the schedule of your events from the admin panel

A setup using these 3 elements will provide everything you need to set up what you're looking for. In terms of implementation, I could try and type it all out here, but I feel I would just be copying the guides I followed. I found this one particularly useful, but I assume you're not using Docker (I recommend you do though!). Either way, between that and the official Celery Getting Started Guide and Django Celery Beat guide , you'll have everything you need.