How to run .py files as cronjob in aws?

Question

I have a Django application which it's deployed to Amazon Elastic Beanstalk(Python 3.7 running on 64bit Amazon Linux 2/3.1.1). I have been trying to run a .py file as a cronjob that works at 4 am every day in AWS and I have created a .config file into .ebextensions for that such as below.

cron.config file:

files:
    "/etc/cron.d/cron_process":
        mode: "000644"
        owner: root
        group: root
        content: |
            0 4 * * * root /usr/local/bin/task_process.sh

    "/usr/local/bin/task_process.sh":
        mode: "000755"
        owner: root
        group: root
        content: |
            #!/bin/bash

            date > /tmp/date
            source /var/app/venv/staging-LQM1lest/bin/activate
            cd /var/app/current
            python Bot/run_spiders.py
            exit 0

commands:
    remove_old_cron:
        command: "rm -f /etc/cron.d/cron_process.bak"

run_spiders.py file:

from first_bot.start import startallSpiders
from .models import Spiders
import importlib

test = Spiders.objects.get(id=1)
test.spider_name = "nigdehalk"
test.save()

I'm trying to change an attribute of one of my objects for testing but it didn't work. Am I missing something? How can I create this cronjob?

Answer 1

Based on the comments.

The issue was caused by failure of task_process.sh . Normally, cron will not show error messages if task_process.sh fail. To help troubleshoot the issue, you can redirect all output into a text file using:

exec &>> /tmp/cron_capture_log.txt

Subsequently, the EB config file could be:

files:
    "/etc/cron.d/cron_process":
        mode: "000644"
        owner: root
        group: root
        content: |
            0 4 * * * root /usr/local/bin/task_process.sh

    "/usr/local/bin/task_process.sh":
        mode: "000755"
        owner: root
        group: root
        content: |
            #!/bin/bash

            exec &>> /tmp/cron_capture_log.txt

            date > /tmp/date
            source /var/app/venv/staging-LQM1lest/bin/activate
            cd /var/app/current
            python Bot/run_spiders.py
            exit 0

commands:
    remove_old_cron:
        command: "rm -f /etc/cron.d/cron_process.bak"

The above helps to identify issues with task_process.sh when running in cron.