Django back-end scripts scheduling

Question

I'm building a website with Django and among other things I want to display the latest news about a certain topic. I have a python script in the back-end that I would like to program to retrieve the latest news once every 1 hour for example. In the meantime I want to be able to display the most recently retrieved news. I'm doing this in order to avoid that the script is being activated every time someone opens my website.

My script is in news.py:

import pprint
import requests
import datetime
import pandas as pd
import dateutil.parser

secret = "********"
url = 'https://newsapi.org/v2/everything?'

quote = 'Amazon'

parameters1 = {
    'q': quote,

    'pageSize': 100,
    'sortby': 'publishedAt',
    'apiKey': secret,
}

response1 = requests.get(url, params=parameters1)
response_json1 = response1.json()
text_combined1 = []
for i in response_json1['articles']:
    if i['content'] != None:
        case = {'Source': i['source']['name'], 'Title': i['title'], 'url': i['url'],
                'Published on:': dateutil.parser.parse(i['publishedAt']).strftime('%Y-%m-%d'), 'Image': i['urlToImage']}
        text_combined1.append(case)
data_amazon = pd.DataFrame.from_dict(text_combined1)
news1 = data_amazon.iloc[0]
news2 = data_amazon.iloc[1]
news3 = data_amazon.iloc[2]

My views.py looks like this:

from django.shortcuts import render
from .news import *

def dashboard(request):
    content = {'data': data, 'news1': news1, 'news2': news2, 'news3': news3}
    return render(request, 'dashboard.html',
                  content)

I'm new to web development but my understanding as of now is that every time someone sends a request to my webpage that script would be run, which would result in delay in the display of the news and most likely access denial to the news.api due to too many requests.

Thank you in advance!

Answer 1

A good way to do this is with Celery. It will let you schedule tasks in Django.

You can read more about it here, and see some other options as well.

Set up a scheduled job?