How to run Rails migrations and seeding in Amazon Elastic Beanstalk single container Docker environment

Question

I'm working on deploying a Rails application to Elastic Beanstalk using docker and so far everything has worked out. I'm at the point where the application needs to run migrations and seeding of the database, and I'm having trouble figuring out exactly how I need to proceed. It appears that any commands in the /.ebextensions folder run in the context of the host machine and not the docker container. Is that correct?

I'm fine with running a command to execute migrations inside of the docker container after startup, but how do I ensure that the migrations only run on a single instance? Is there an environment variable or some other way I can tell what machine is the leader from within the docker container?

---

Update: I posted a question in the Amazon Elastic Beanstalk forums asking how to run "commands from Docker host on the container" on the 6th/Aug/15'. You can follow the conversations there as well as they are useful.

Answer 1

I'm not sure the solution you have proposed is going to work. It appears that the current process for EB Docker deployment runs container commands before the new docker container is running, which means that you can't use docker exec on it. I suspect that your commands will execute against the old container which is not yet taken out of service.

After much trial and error I got this working through using container commands with a shell script.

container_commands:
  01_migrate_db:
    command: ".ebextensions/scripts/migrate_db.sh"
    leader_only: true

And the script:

if [ "${PROCESS}" = "WEB" ]; then

  . /opt/elasticbeanstalk/hooks/common.sh

  EB_SUPPORT_FILES=$(/opt/elasticbeanstalk/bin/get-config container -k support_files_dir)

  EB_CONFIG_DOCKER_ENV_ARGS=()

  while read -r ENV_VAR; do
    EB_CONFIG_DOCKER_ENV_ARGS+=(--env "$ENV_VAR")
  done < <($EB_SUPPORT_FILES/generate_env)

  echo "Running migrations for aws_beanstalk/staging-app"
  docker run --rm "${EB_CONFIG_DOCKER_ENV_ARGS[@]}" aws_beanstalk/staging-app bundle exec rake db:migrate || echo "The Migrations failed to run."
fi
true

I wrap the whole script in a check to ensure that migrations don't run on background workers.

I then build the ENV in exactly the same way that EB does when starting the new container so that the correct environment is in place for the migrations.

Finally I run the command against the new container which has been created but is not yet running - aws_beanstalk/staging-app . It exits at the end of the migration and the --rm removes the container automatically.

Answer 2

Update: This solution, though seemingly correct, doesn't work as intended (it seemed it was at first though). For reasons best explained in nmott 's answer below. Will leave it here for posterity.

---

I was able to get this working using container_commands via the .ebextensions directory config files. Learn more about container commands here. And I quote ...

The commands in container_commands are processed in alphabetical order by name. They run after the application and web server have been set up and the application version file has been extracted, but before the application version is deployed. They also have access to environment variables such as your AWS security credentials. Additionally, you can use leader_only. One instance is chosen to be the leader in an Auto Scaling group. If the leader_only value is set to true, the command runs only on the instance that is marked as the leader.

So, applying that knowledge ... the container_commands.config will be ...

# .ebextensions/container_commands.config
container_commands:
  01_migrate_db:
    command: docker exec `docker ps -l -q -f 'status=running'` rake db:migrate RAILS_ENV=production
    leader_only: true
    ignoreErrors: false
  02_seed_db:
    command: docker exec `docker ps -l -q -f 'status=running'` rake db:seed RAILS_ENV=production
    leader_only: true
    ignoreErrors: false

That runs the migrations first and then seeds the database. We use docker exec [OPTIONS] CONTAINER_ID COMMAND [ARG...] which runs the appended COMMAND [ARG...] in the context of the existing container (not the host). And we get CONTAINER_ID by running docker ps -q .

Answer 3

Solution 1: run migration when you start server

In the company I work for we have literally equivalent for this line to start the production server:

 bundle exec rake db:migrate && bundle exec puma -C /app/config/puma.rb

• https://github.com/equivalent/docker_rails_aws_elasticbeanstalk_demmo_app/blob/master/puppies/script/start_server.sh .:

• https://github.com/equivalent/docker_rails_aws_elasticbeanstalk_demmo_app/blob/master/puppies/Dockerfile

And yes this is Load balanced environment (3 - 12 instances depending on load) and yes they all execute this script. (we do load balance by introducing 1 instance at a time during deployment)

The thing is the first batch of deployment (first instance up ) will execute the bundle exec rake db:migrate and run the migrations (meaning it will run the DB changes) and then once done it will run the server bundle exec puma -C /app/config/puma.rb

The second deployment batch (2nd instance) will also run the bundle exec rake db:migrate but will not do anything (as there are no pending migrations). It will just continue to the second part of the script bundle exec puma -C /app/config/puma.rb o

So honestly I don't think this is the perfect solution but is pragmatic and works for our team I don't believe there is any generic "best practice" for EB out there for Rails running migrations as some application teams don't want to run the migrations after the deployment while others (like our team) they do want to run them straight after deployment.

Solution 2: background worker Enviromnet to run migrations

if you have Worker like Delayed job, Sidekiq, Rescue on own EB enviroment you can configure them to run the migrations:

bundle exec rake db:migrate && bundle exec sidekiq )

So first you willdeploy the worker and once the worker is deployed then deploy webserver that will not run the migrations

eg: just bundle exec puma

Solution 3 Hooks

I agree that using EB hoos ore ok far this but honestly I use eb hooks only for more complex devops stuff (like pulling ssl certificates for the Nginx web-server) not for running migrations)

anyway hooks were already covered in this SO question so I'll not repeat the solution. I will just reference this article that will help you understand them:

• https://blog.eq8.eu/article/aws-elasticbeanstalk-hooks.html

Conclusion

It's really up to you to figure out what is the best for your application. But honestly EB is really simple tool (compared to tools like Ansible or Kubernetes) No mater what you implement as long as it works its ok :)

One more helpful link for EB for Rails developers:

• talk: AWS Elastic Beanstalk & Docker for Rails developers

Answer 4

Use .ebextensions/01-environment.config :

container_commands:
  01_write_leader_marker:
    command: touch /tmp/is_leader
    leader_only: true

Now add directory /tmp to volumes in Dockerfile / Dockerrun.aws.json.

Then check set all initialization commands like db migration in sh script that first check if file /tmp/is_leader exists and executes them only in this case.