Make rails and sidekiq work together from different Docker containers (but can't use docker-compose)

Question

I'm moving a rails app from Heroku to a linux server and deploying it using Caprover. It's an app very dependent on background jobs, which I run with sidekiq.

I've managed to make it work by running both the rails server ( bundle exec rails server -b 0.0.0.0 -p80 & ) and sidekiq ( bundle exec sidekiq & ) from a script that launches both in the CMD of the Dockerfile.

But I guess it would be much better (separation of concerns) if the rails server was in one Docker container and sidekiq in another one. But I can't figure out how to connect them. How do I tell my rails app that sidekiq lives in another container?

Because I use Caprover I'm limited to Dockerfiles to deploy my images, so I can't use docker-compose .

Is there a way to tell rails that it should use a certain sidekiq found in a certain Docker container? Caprover uses Docker swarm if that is of any help.

Am I thinking about this the wrong way?

My setup, currently, is as follows:

• 1 Docker container with rails server + sidekiq
• 1 Docker container with the postgres DB
• 1 Docker container with the Redis DB

My desired setup would be:

• 1 Docker container with rails server
• 1 Docker container with sidekiq
• 1 Docker container with postgres DB
• 1 Docker container with Redis DB

Is that even possible with my current limitations?

My rails + sidekiq Dockerfile is as follows:

FROM ruby:2.6.4-alpine
#
RUN apk update && apk add nodejs yarn postgresql-client postgresql-dev tzdata build-base ffmpeg
RUN apk add --no-cache --upgrade bash
RUN mkdir /myapp
WORKDIR /myapp
COPY Gemfile /myapp/Gemfile
COPY Gemfile.lock /myapp/Gemfile.lock
RUN bundle install --deployment --without development test
COPY . /myapp
RUN yarn 
RUN bundle exec rake yarn:install
# Set production environment
ENV RAILS_ENV production
ENV RAILS_SERVE_STATIC_FILES true
# Assets, to fix missing secret key issue during building
RUN SECRET_KEY_BASE=dumb bundle exec rails assets:precompile
# Add a script to be executed every time the container starts.
COPY entrypoint.sh /usr/bin/
RUN chmod +x /usr/bin/entrypoint.sh
ENTRYPOINT ["entrypoint.sh"]
EXPOSE 80
COPY start_rails_and_sidekiq.sh /myapp/start_rails_and_sidekiq.sh
RUN chmod +x /myapp/start_rails_and_sidekiq.sh
# Start the main process.
WORKDIR /myapp
CMD ./start_rails_and_sidekiq.sh

the start_rails_and_sidekiq.sh looks like this:

#!/bin/bash

# Start the first process
bundle exec rails server -b 0.0.0.0 -p80 &
status=$?
if [ $status -ne 0 ]; then
  echo "Failed to start Rails server: $status"
  exit $status
fi

# Start the second process
bundle exec sidekiq &
status=$?
if [ $status -ne 0 ]; then
  echo "Failed to start Sidekiq: $status"
  exit $status
fi

# Naive check runs checks once a minute to see if either of the processes exited.
# This illustrates part of the heavy lifting you need to do if you want to run
# more than one service in a container. The container exits with an error
# if it detects that either of the processes has exited.
# Otherwise it loops forever, waking up every 60 seconds

while sleep 60; do
  ps aux |grep puma |grep -q -v grep
  PROCESS_1_STATUS=$?
  ps aux |grep sidekiq |grep -q -v grep
  PROCESS_2_STATUS=$?
  # If the greps above find anything, they exit with 0 status
  # If they are not both 0, then something is wrong
  if [ $PROCESS_1_STATUS -ne 0 -o $PROCESS_2_STATUS -ne 0 ]; then
    echo "One of the processes has already exited."
    exit 1
  fi
done

I'm totally lost! Thanks in advance!

Answer 1

Method 1

According to CapRover docs , it seems that it is possible to run Docker Compose on CapRover, but I haven't tried it myself (yet).

Method 2

Although this CapRover example is for a different web app, the Internal Access principle is the same:

You can simply add a srv-captain-- prefix to the name of the container if you want to access it from another container.

However, isn't this method how you told your Rails web app where to find the PostgreSQL DB container? Or are you accessing it through an external subdomain name?