[英]How do I link a dockerized python script to a mysql docker container to load data on the same host?
I have two docker containers running in an Ubuntu 16.04 machine, one docker container has a mysql
sever running, the other container holds a dockerized python
script set to run a cron job every minute that loads data into mysql
.我有两个 docker 容器在 Ubuntu 16.04 机器上运行,一个 docker 容器运行一个
mysql
服务器,另一个容器包含一个 dockerized python
脚本,设置为每分钟运行一个 cron 作业,将数据加载到mysql
。 How can I connect the two to load data through the python script into the mysql container?如何将两者连接起来通过python脚本将数据加载到mysql容器中? I have an error showing up: Here are my relevant commands:
我有一个错误显示:这是我的相关命令:
MYSQL container runs without issue: MYSQL 容器运行没有问题:
docker run -p 3306:3306 -e MYSQL_ROOT_PASSWORD=yourPassword --name icarus -d mysql_docker_image
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
927e50ca0c7d mysql_docker_image "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:3306->3306/tcp, 33060/tcp icarus
Second container holds cron and python script:第二个容器包含 cron 和 python 脚本:
#build the container without issue
sudo docker run -t -i -d docker-cron
#exec into it to check logs
sudo docker exec -i -t container_id /bin/bash
#check logs
root@b149b5e7306d:/# cat /var/log/cron.log
Error:错误:
have the following error showing up, which I believe has to do with wrong host address:出现以下错误,我认为这与错误的主机地址有关:
Caught this error: OperationalError('(pymysql.err.OperationalError) (2003, "Can\'t connect to MySQL server on \'localhost\' ([Errno 99] Cannot assign requested address)")',)
Python Script: Python脚本:
from traffic.data import opensky
from sqlalchemy import create_engine
#from sqlalchemy_utils import database_exists, create_database
import sqlalchemy
import gc
#connection and host information
host = 'localhost'
db='icarus'
engine = create_engine('mysql+pymysql://root:password@'+ host+ ':3306/'+ db) #create engine connection
version= sys.version_info[0]
#functions to upload data
def upload(df,table_name):
df.to_sql(table_name,con=engine,index=False,if_exists='append')
engine.dispose()
print('SUCCESSFULLY LOADED DATA INTO STAGING...')
#pull data drom api
sv = opensky.api_states()
final_df = sv.data
#quick column clean up
print(final_df.head())
final_df=final_df.rename(columns = {'timestamp':'time_stamp'})
#insert data to staging
try:
upload(final_df, 'flights_stg')
except Exception as error:
print('Caught this error: ' + repr(error))
del(final_df)
gc.collect()
I'm assuming the error is the use of 'localhost' as my address?我假设错误是使用“localhost”作为我的地址? How would i go about resolving something like this?
我将如何解决这样的事情?
More information:更多信息:
MYSQL Dockerfile: MYSQL Dockerfile:
FROM mysql
COPY init.sql /docker-entrypoint-initdb.d
Python Dockerfile: Python Dockerfile:
FROM ubuntu:latest
WORKDIR /usr/src/app
#apt-get install -y build-essential -y python python-dev python-pip python-virtualenv libmysqlclient-dev curl&& \
RUN \
apt-get update && \
apt-get install -y build-essential -y git -y python3.6 python3-pip libproj-dev proj-data proj-bin libgeos++-dev libmysqlclient-dev python-mysqldb curl&& \
rm -rf /var/lib/apt/lists/*
COPY requirements.txt ./
RUN pip3 install --upgrade pip && \
pip3 install --no-cache-dir -r requirements.txt
RUN pip3 install --upgrade setuptools
RUN pip3 install git+https://github.com/xoolive/traffic
COPY . .
# Install cron
RUN apt-get update
RUN apt-get install cron
# Add crontab file in the cron directory
ADD crontab /etc/cron.d/simple-cron
# Add shell script and grant execution rights
ADD script.sh /script.sh
RUN chmod +x /script.sh
# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/simple-cron
# Create the log file to be able to run tail
RUN touch /var/log/cron.log
# Run the command on container startup
CMD cron && tail -f /var/log/cron.log
Can you share your dockerfile or compose file from MySQL container.您可以从 MySQL 容器共享您的 dockerfile 或撰写文件吗? Yes, the problem related to using localhost as host.
是的,与使用 localhost 作为主机有关的问题。 You must use a docker service name as host.
您必须使用 docker 服务名称作为主机。 So in docker service name works as DNS.
所以在 docker 服务名称作为 DNS。 For example if your docker-compose looks like:
例如,如果您的 docker-compose 如下所示:
services:
mydb:
image: mysql:5.7
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_ROOT_PASSWORD:root
MYSQL_USER: root
MYSQL_PASSWORD: root
MYSQL_DATABASE: root
You must use mydb
instead of localhost您必须使用
mydb
而不是 localhost
Looks like a user defined network bridge was the way to go here as recommended.看起来用户定义的网桥是推荐的方式。 Solved issue by:
通过以下方式解决问题:
docker network create my-net
docker create --name mysql \
--network my-net \
--publish 3306:3306 \
-e MYSQL_ROOT_PASSWORD=password \
mysql:latest
docker create --name docker-cron \
--network my-net \
docker-cron:latest
docker start each of them and using the --name as the host was perfect. docker 启动它们中的每一个并使用 --name 作为主机是完美的。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.