Mysql2::Error::ConnectionError: Can't connect to MySQL server on 'mysql' (111 "Connection refused")

Question

i am running the docker-compose for my sample applications, and this file consist of mysql database and the application image. i am getting the following error.

    $ docker logs -f sample-application
    Can't connect to MySQL server on 'mysql' (111 "Connection refused")
    Couldn't create database for {"adapter"=>"mysql2", "encoding"=>"utf8", "pool"=>5, "reconnect"=>true, "database"=>"datagen", "username"=>"root", "password"=>"password", "host"=>"mysql", "port"=>3306}
    rake aborted!
    Mysql2::Error::ConnectionError: Can't connect to MySQL server on 'mysql' (111 "Connection refused")
    /usr/local/bundle/gems/mysql2-0.5.2/lib/mysql2/client.rb:90:in `connect'
    /usr/local/bundle/gems/mysql2-0.5.2/lib/mysql2/client.rb:90:in `initialize'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/connection_adapters/mysql2_adapter.rb:22:in `new'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/connection_adapters/mysql2_adapter.rb:22:in `mysql2_connection'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:809:in `new_connection'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:853:in `checkout_new_connection'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:832:in `try_to_checkout_new_connection'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:793:in `acquire_connection'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:521:in `checkout'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:380:in `connection'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:1008:in `retrieve_connection'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/connection_handling.rb:118:in `retrieve_connection'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/connection_handling.rb:90:in `connection'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/tasks/mysql_database_tasks.rb:6:in `connection'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/tasks/mysql_database_tasks.rb:14:in `create'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/tasks/database_tasks.rb:119:in `create'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/tasks/database_tasks.rb:139:in `block in create_current'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/tasks/database_tasks.rb:316:in `block in each_current_configuration'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/tasks/database_tasks.rb:313:in `each'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/tasks/database_tasks.rb:313:in `each_current_configuration'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/tasks/database_tasks.rb:138:in `create_current'
    /usr/local/bundle/gems/activerecord-5.2.1/lib/active_record/railties/databases.rake:29:in `block (2 levels) in <top (required)>'
    /usr/local/bundle/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
    Tasks: TOP => db:create
    (See full trace by running task with --trace)

Here is database components for mysql docker-compose yaml file

version: '3.1'

services:
sample-service:
    image: service:latest
    hostname: sample-service
    container_name: sample-service
    depends_on:
      - sample-application
    ports:
      - "8080:8080"
    networks:
      - $NETWORK_NAME

mysql:
    image: mysql:5.7
    hostname: mysql
    container_name: mysql
    ports:
      - "3306:3306"
    environment:
      - MYSQL_ROOT_PASSWORD=password
      - MYSQL_DATABASE=foo
      - MYSQL_USER=root
      - MYSQL_PASSWORD=password
    networks:
      - $NETWORK_NAME

sample-application:
    image: sample:latest 
    hostname: mysql
    container_name: sample-application
    depends_on:
      - mysql
    ports:
      - "3000:3000"
    environment:
      - RAILS_ENV=production
      - DATABASE_HOST=mysql
      - DATABASE_USER=root
      - DATABASE_NAME=foo
      - DATABASE_PORT=3306
    networks:
      - $NETWORK_NAME

networks:
  datagen:
    external: true

I am able to ping mysql from application docker container image. The connection is successfull and at the same time i am able to access the mysql image from some other machine also.

So there is no issue in the mysql image but i am unable to connect to mysql host when running using docker compose. The environmental variable which i passes is also correct.

I looked into various issues in stackoverflow and tried to change the bind address. But i dont see it in my mysql container image.

    bash-4.2# find / -name "*.cnf"
    /var/lib/mysql/auto.cnf
    /usr/share/mysql/my-default.cnf
    /etc/pki/tls/openssl.cnf
    /etc/my.cnf
    /healthcheck.cnf
    bash-4.2# cat /etc/my.cnf
    # For advice on how to change settings please see
    # http://dev.mysql.com/doc/refman/5.6/en/server-configuration-defaults.html

    [mysqld]
    #
    # Remove leading # and set to the amount of RAM for the most important data
    # cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
    # innodb_buffer_pool_size = 128M
    #
    # Remove leading # to turn on a very important data integrity option: logging
    # changes to the binary log between backups.
    # log_bin
    #
    # Remove leading # to set options mainly useful for reporting servers.
    # The server defaults are faster for transactions and fast SELECTs.
    # Adjust sizes as needed, experiment to find the optimal values.
    # join_buffer_size = 128M
    # sort_buffer_size = 2M
    # read_rnd_buffer_size = 2M
    skip-host-cache
    skip-name-resolve
    datadir=/var/lib/mysql
    socket=/var/lib/mysql/mysql.sock
    secure-file-priv=/var/lib/mysql-files
    user=mysql

    # Disabling symbolic-links is recommended to prevent assorted security risks
    symbolic-links=0

    log-error=/var/log/mysqld.log
    pid-file=/var/run/mysqld/mysqld.pid

Answer 1

You really need external network? The only possible reason is that you want to connect other containers to it. Otherwise don't use it - you will be able to access containers from host just through port exposal mechanism.

If you are not going to access database from outside docker-compose , consider following changes to file:

• removed useless stuff
• removed ports (inside user-defined network you don't need exposing)

docker-compose.yml

version: '3.1'

services:
  mysql:
    image: mysql:5.7
    environment:
      - MYSQL_ROOT_PASSWORD=password
      - MYSQL_DATABASE=foo
      - MYSQL_USER=root
      - MYSQL_PASSWORD=password

  sample-application:
    image: sample:latest 
    depends_on:
      - mysql
    environment:
      - RAILS_ENV=production
      - DATABASE_HOST=mysql
      - DATABASE_USER=root
      - DATABASE_NAME=foo
      - DATABASE_PORT=3306

  sample-service:
    image: service:latest
    depends_on:
      - sample-application

With such configuration you can access services inside network by their names, eg mysql , sample-service and sample-application

Btw strange situation when service depends on application and not vice versa

Answer 2

If you are using rails to connect to Mysql, make sure the db host in database.yml is same as the name of the service(in this case mysql ) in docker-compose , that is how it knows to connect to mysql container. Remove all containers and images and build from beginning just to make sure no configuration is cached.