Django BaseCommand works locally but does not work on Amazon AWS EB

Question

So, I am in the process of deploying a Django website using Amazon AWS EB + RDS database.

It is all working fine except from a script that I have that is in charge of populating the database programmatically. I managed to have it running locally following this very nice and short tutorial: http://eli.thegreenplace.net/2014/02/15/programmatically-populating-a-django-database So locally it works as it is supposed to.

My problem comes when I run this script on my amazon AWS EB (both with cron or manually with my env activated).

I get this error message:

'''
Traceback (most recent call last):
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
    self.connect()
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/db/backends/base/base.py", line 189, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/db/backends/postgresql/base.py", line 176, in get_new_connection
    connection = Database.connect(**conn_params)
  File "/opt/python/run/venv/local/lib64/python3.4/site-packages/psycopg2/__init__.py", line 130, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not connect to server: Connection refused
    Is the server running on host "localhost" (127.0.0.1) and accepting
    TCP/IP connections on port 5432?


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/python/current/app/BetzCenter/manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/core/management/__init__.py", line 363, in execute_from_command_line
    utility.execute()
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/core/management/__init__.py", line 355, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/opt/python/bundle/18/app/BetzCenter/gamestream/management/commands/odds.py", line 444, in handle
    self.create_gamepages(self.all_fixtures())
  File "/opt/python/bundle/18/app/BetzCenter/gamestream/management/commands/odds.py", line 433, in create_gamepages
    for fixture in fixtures:
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/db/models/query.py", line 250, in __iter__
    self._fetch_all()
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/db/models/query.py", line 1118, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 882, in execute_sql
    cursor = self.connection.cursor()
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/db/backends/base/base.py", line 254, in cursor
    return self._cursor()
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/db/backends/base/base.py", line 229, in _cursor
    self.ensure_connection()
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
    self.connect()
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
    self.connect()
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/db/backends/base/base.py", line 189, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/opt/python/run/venv/local/lib/python3.4/site-packages/django/db/backends/postgresql/base.py", line 176, in get_new_connection
    connection = Database.connect(**conn_params)
  File "/opt/python/run/venv/local/lib64/python3.4/site-packages/psycopg2/__init__.py", line 130, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError: could not connect to server: Connection refused
    Is the server running on host "localhost" (127.0.0.1) and accepting
    TCP/IP connections on port 5432?
'''

The script runs fine (on AWS EB) if I manually use psycopg2 code with sql statements, but fails to use Django settings which I thought I add applied following that tutorial that I linked.

I really want to be able to use django commands for this script though.

Any ideas on how to debug this? Googling hasn't really helped me too much for now.

UPDATE --> I was pretty sure my "normal" python script is not pointing to the right DB. In fact, I have now tried to print the database it points to and I was right. It does not point to my amazon RDS db (but looks for a local one), while the rest of my Django app does. I have tried the solutions that I found here and there on stackoverflow. Where should I start debugging this? ;)

Thanks! Vittorio

Answer 1

So I got back to this after a few months and managed to sort it. I was a matter of variables. After some research and after trying a few solutions, this is what worked for me:

I added this in the crontab file, where I call the command (after activating the virtual environment and before running the command itself:

&& source /opt/python/current/env

If I got it correctly this is specific to AWS elastick beanstalk and passed all the right variables so that my command does not fail. The whole crontab line now looks like:

0,30 * * * * source /opt/python/run/venv/bin/activate && source /opt/python/current/env && python /opt/python/current/app/my_site/manage.py my_command

Hope this can help.