[英]docker-compose Unknown MySQL server host 'mysql' (airflow)
I try to create my Airflow service. 我尝试创建我的Airflow服务。 When I run three dockers separately (without
docker-compose
) everything works but when I try to do the same using docker-compose
I get the error. 当我分别运行三个码头
docker-compose
(没有docker-compose
)时,一切正常,但是当我尝试使用docker-compose
做同样的事情时,我得到了错误。 It seems that airflow doesn't see database. 似乎气流没有看到数据库。
docker-compose.yml
version: '2'
services:
mysql:
image: "someregisty/mysql"
environment:
- MYSQL_ROOT_PASSWORD=somepassword
redis:
image: "someregisty/redis"
airflow:
image: "someregisty/airflow"
volumes:
- ./dags:/home/airflow/dags
- ./logs:/home/airflow/logs
- ./config/airflow.cfg:/home/airflow/airflow.cfg
- ./config/resolv.conf:/etc/resolv.conf
- ./config/somekeytab.keytab:/home/airflow/airflow.keytab
ports:
- "8484:8080"
environment:
- AIRFLOW__CELERY__BROKER_URL=redis://redis:6379/0
- AIRFLOW__CELERY__CELERY_RESULT_BACKEND=db+mysql://root:somepassword@mysql/airflow
- AIRFLOW__CORE__SQL_ALCHEMY_CONN=mysql://root:somepassword@mysql/airflow
- AIRFLOW__CORE__AIRFLOW_HOME=/home/airflow
- AIRFLOW__CORE__DAGS_FOLDER=/home/airflow/dags/
- AIRFLOW__CORE__PLUGINS_FOLDER=/home/airflow/plugins/
- AIRFLOW_SCHEDULER=true
- AIRFLOW_WEBSERVER=true
- AIRFLOW_WORKER=true
- AIRFLOW_KERBEROS=true
- AIRFLOW_KERBEROS_PRINCIPAL=prj_bdc@DCWP.PL
- AIRFLOW__KERBEROS__CCACHE=/home/airflow/kerberos.keytab.cache
- KRB5CCNAME=/home/airflow/kerberos.keytab.cache
depends_on:
- redis
- mysql
I run 我跑
docker-compose up -d
docker-compose exec mysql mysql -u root --password=somepassword -e "create database airflow"
and that works ok. 这没关系。 Then I run
然后我跑了
docker-compose exec airflow airflow initdb
And I get 我明白了
[2017-09-28 16:34:36,100] {__init__.py:57} INFO - Using executor CeleryExecutor
/usr/lib/python2.7/site-packages/airflow/utils/helpers.py:406: DeprecationWarning: Importing BaseSensorOperator directly from <module 'airflow.operators' from '/usr/lib/python2.7/site-packages/airflow/operators/__init__.pyc'> has been deprecated. Please import from '<module 'airflow.operators' from '/usr/lib/python2.7/site-packages/airflow/operators/__init__.pyc'>.[operator_module]' instead. Support for direct imports will be dropped entirely in Airflow 2.0.
DeprecationWarning)
DB: mysql://root:***@mysql/airflow
[2017-09-28 16:34:36,666] {db.py:287} INFO - Creating tables
Traceback (most recent call last):
File "/usr/bin/airflow", line 28, in <module>
args.func(args)
File "/usr/lib/python2.7/site-packages/airflow/bin/cli.py", line 951, in initdb
db_utils.initdb()
File "/usr/lib/python2.7/site-packages/airflow/utils/db.py", line 106, in initdb
upgradedb()
File "/usr/lib/python2.7/site-packages/airflow/utils/db.py", line 294, in upgradedb
command.upgrade(config, 'heads')
File "/usr/lib/python2.7/site-packages/alembic/command.py", line 174, in upgrade
script.run_env()
File "/usr/lib/python2.7/site-packages/alembic/script/base.py", line 416, in run_env
util.load_python_file(self.dir, 'env.py')
File "/usr/lib/python2.7/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
module = load_module_py(module_id, path)
File "/usr/lib/python2.7/site-packages/alembic/util/compat.py", line 79, in load_module_py
mod = imp.load_source(module_id, path, fp)
File "/usr/lib/python2.7/site-packages/airflow/migrations/env.py", line 86, in <module>
run_migrations_online()
File "/usr/lib/python2.7/site-packages/airflow/migrations/env.py", line 73, in run_migrations_online
with connectable.connect() as connection:
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2091, in connect
return self._connection_cls(self, **kwargs)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 90, in __init__
if connection is not None else engine.raw_connection()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2177, in raw_connection
self.pool.unique_connection, _connection)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2151, in _wrap_pool_connect
e, dialect, self)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1465, in _handle_dbapi_exception_noconnection
exc_info
File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2147, in _wrap_pool_connect
return fn()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 328, in unique_connection
return _ConnectionFairy._checkout(self)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 766, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 516, in checkout
rec = pool._do_get()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1138, in _do_get
self._dec_overflow()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1135, in _do_get
return self._create_connection()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 333, in _create_connection
return _ConnectionRecord(self)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 461, in __init__
self.__connect(first_connect_check=True)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 651, in __connect
connection = pool._invoke_creator(self)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 105, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 393, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/lib64/python2.7/site-packages/MySQLdb/__init__.py", line 86, in Connect
return Connection(*args, **kwargs)
File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 204, in __init__
super(Connection, self).__init__(*args, **kwargs2)
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (2005, "Unknown MySQL server host 'mysql' (0)")
I don't know what is wrong. 我不知道出了什么问题。 Maybe you could help me?
也许你可以帮助我? :)
:)
Versions of docker and docker-compose are docker和docker-compose的版本是
Docker version 1.12.6, build 96d83a5/1.12.6
docker-compose version 1.9.0, build 2585387
The problem is caused by binding resolv.conf
file into airflow container. 该问题是由
resolv.conf
文件绑定到气流容器引起的。 The reason you get unknown MySQL server host 'mysql'
error from airflow container is that mysql
hostname does not resolve to the ip of airflow container. 你从气流容器中得到
unknown MySQL server host 'mysql'
错误的原因是mysql
主机名没有解析到气流容器的ip。
Remove ./config/resolv.conf:/etc/resolv.conf
from Compose configuration . 从Compose配置中删除
./config/resolv.conf:/etc/resolv.conf
。 It will cause Docker to resolve hostname correctly. 它将导致Docker正确解析主机名。 If you need to specify additional DNS servers use Compose
--dns
option documented here . 如果需要指定其他DNS服务器,请使用此处记录的 Compose
--dns
选项。
Add nameserver 127.0.0.11
line to /etc/resolv.conf
file . 将
nameserver 127.0.0.11
行添加到/etc/resolv.conf
文件中 。 It enables usage of Docker embedded DNS server, which is a default setup. 它支持使用Docker嵌入式DNS服务器,这是一种默认设置。 You can keep your custom entries in that file as well.
您也可以将自定义条目保留在该文件中。 Note that this solution is a hack because, as stated in documentation, Docker is responsible for managing
/etc/resolv.conf
file. 请注意,此解决方案是一种黑客攻击,因为如文档中所述,Docker负责管理
/etc/resolv.conf
文件。 Quote from documentation to back it up: 从文档引用备份:
So you should not assume the way the files such as /etc/hosts, /etc/resolv.conf are managed inside the containers and leave the files alone and use the following Docker options instead
因此,您不应该假设/ etc / hosts,/ etc / resolv.conf等文件在容器内部进行管理并保留文件,并使用以下Docker选项代替
You can read more about Docker embedded DNS server in user-defined networks here . 您可以在此处阅读有关用户定义网络中的Docker嵌入式DNS服务器的更多信息。
So the error is a bit misleading. 所以这个错误有点误导。 The connect is failing because of the DB not being there, not that mysql host is not there.
连接失败是因为DB不存在,而不是mysql主机不在那里。
Change your mysql service to below 将您的mysql服务更改为以下
mysql:
image: "someregisty/mysql"
environment:
- MYSQL_ROOT_PASSWORD=somepassword
- MYSQL_DATABASE=airflow
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.