[英]Using AWS EFS with Docker
I am using the new Elastic File System provided by amazon, on my single container EB deploy. 我在我的单个容器EB部署中使用亚马逊提供的新弹性文件系统。 I can't figure out why the mounted EFS cannot be mapped into the container.
我无法弄清楚为什么挂载的EFS无法映射到容器中。
The EFS mount is successfully performed on the host at /efs-mount-point. EFS挂载已在/ efs-mount-point上的主机上成功执行。
Provided to the Dockerrun.aws.json is 提供给Dockerrun.aws.json是
{
"AWSEBDockerrunVersion": "1"
"Volumes": [
{
"HostDirectory": "/efs-mount-point",
"ContainerDirectory": "/efs-mount-point"
}
]
}
The volume is then created in the container once it starts running. 然后,一旦卷开始运行,就会在容器中创建卷。 However it has mapped the hosts directory /efs-mount-point, not the actual EFS mount point.
但是它已映射hosts目录/ efs-mount-point,而不是实际的EFS挂载点。 I can't figure out how to get Docker to map in the EFS volume mounted at /efs-mount-point instead of the host's directory.
我无法弄清楚如何让Docker映射到安装在/ efs-mount-point而不是主机目录的EFS卷中。
Do NFS volumes play nice with Docker? NFS卷与Docker一起玩得好吗?
You need to restart docker after mounting the EFS volume in the host EC2 instance. 在主机EC2实例中安装EFS卷后,需要重新启动docker 。
Here's an example, .ebextensions/efs.config
: 这是一个例子,
.ebextensions/efs.config
:
commands:
01mkdir:
command: "mkdir -p /efs-mount-point"
02mount:
command: "mountpoint -q /efs-mount-point || mount -t nfs4 -o nfsvers=4.1 $(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone).fs-fa35c253.efs.us-west-2.amazonaws.com:/ /efs-mount-point"
03restart:
command: "service docker restart"
EFS with AWS Beanstalk - Multicontainer Docker will work. 使用AWS Beanstalk的EFS - Multicontainer Docker将工作。 But numerous of things will stop working because you have to restart docker after you mount the EFS.
但是许多事情都会停止工作,因为你必须在安装EFS后重新启动docker。
Searching around you might find that you need to do "docker restart" after mounting EFS. 在您周围搜索可能会发现在安装EFS后需要执行“docker restart”。 It's not that simple.
这不是那么简单。 You will experience troubles when autoscaling happens and / or when deploying new version of the app.
当自动调节发生和/或部署新版本的应用程序时,您将遇到麻烦。
Below is a script I use for mounting a EFS to the docker instance, where the following steps is needed: 下面是我用于将EFS安装到docker实例的脚本,其中需要执行以下步骤:
Here is my script: 这是我的脚本:
.ebextensions/commands.config
: .ebextensions/commands.config
:
commands:
01stopdocker:
command: "sudo stop ecs > /dev/null 2>&1 || /bin/true && sudo service docker stop"
02killallnetworkbindings:
command: 'sudo killall docker > /dev/null 2>&1 || /bin/true'
03removenetworkinterface:
command: "rm -f /var/lib/docker/network/files/local-kv.db"
test: test -f /var/lib/docker/network/files/local-kv.db
# Mount the EFS created in .ebextensions/media.config
04mount:
command: "/tmp/mount-efs.sh"
# On new instances, delay needs to be added because of 00task enact script. It tests for start/ but it can be various states of start...
# Basically, "start ecs" takes some time to run, and it runs async - so we sleep for some time.
# So basically let the ECS manager take it's time to boot before going on to enact scritps and post deploy scripts.
09restart:
command: "service docker start && sudo start ecs && sleep 120s"
.ebextensions/mount-config.config
# efs-mount.config
# Copy this file to the .ebextensions folder in the root of your app source folder
option_settings:
aws:elasticbeanstalk:application:environment:
EFS_REGION: '`{"Ref": "AWS::Region"}`'
# Replace with the required mount directory
EFS_MOUNT_DIR: '/efs_volume'
# Use in conjunction with efs_volume.config or replace with EFS volume ID of an existing EFS volume
EFS_VOLUME_ID: '`{"Ref" : "FileSystem"}`'
packages:
yum:
nfs-utils: []
files:
"/tmp/mount-efs.sh":
mode: "000755"
content : |
#!/bin/bash
EFS_REGION=$(/opt/elasticbeanstalk/bin/get-config environment | jq -r '.EFS_REGION')
EFS_MOUNT_DIR=$(/opt/elasticbeanstalk/bin/get-config environment | jq -r '.EFS_MOUNT_DIR')
EFS_VOLUME_ID=$(/opt/elasticbeanstalk/bin/get-config environment | jq -r '.EFS_VOLUME_ID')
echo "Mounting EFS filesystem ${EFS_DNS_NAME} to directory ${EFS_MOUNT_DIR} ..."
echo 'Stopping NFS ID Mapper...'
service rpcidmapd status &> /dev/null
if [ $? -ne 0 ] ; then
echo 'rpc.idmapd is already stopped!'
else
service rpcidmapd stop
if [ $? -ne 0 ] ; then
echo 'ERROR: Failed to stop NFS ID Mapper!'
exit 1
fi
fi
echo 'Checking if EFS mount directory exists...'
if [ ! -d ${EFS_MOUNT_DIR} ]; then
echo "Creating directory ${EFS_MOUNT_DIR} ..."
mkdir -p ${EFS_MOUNT_DIR}
if [ $? -ne 0 ]; then
echo 'ERROR: Directory creation failed!'
exit 1
fi
chmod 777 ${EFS_MOUNT_DIR}
if [ $? -ne 0 ]; then
echo 'ERROR: Permission update failed!'
exit 1
fi
else
echo "Directory ${EFS_MOUNT_DIR} already exists!"
fi
mountpoint -q ${EFS_MOUNT_DIR}
if [ $? -ne 0 ]; then
AZ=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone)
echo "mount -t nfs4 -o nfsvers=4.1 ${AZ}.${EFS_VOLUME_ID}.efs.${EFS_REGION}.amazonaws.com:/ ${EFS_MOUNT_DIR}"
mount -t nfs4 -o nfsvers=4.1 ${AZ}.${EFS_VOLUME_ID}.efs.${EFS_REGION}.amazonaws.com:/ ${EFS_MOUNT_DIR}
if [ $? -ne 0 ] ; then
echo 'ERROR: Mount command failed!'
exit 1
fi
else
echo "Directory ${EFS_MOUNT_DIR} is already a valid mountpoint!"
fi
echo 'EFS mount complete.'
You will have to change the option_settings below. 您将不得不更改下面的option_settings。 To find the VPC and subnets which you must define under option_settings below, look in AWS web console -> VPC, there you must find the Default VPC id and the 3 default subnet ids.
要查找必须在下面的option_settings下定义的VPC和子网,请查看AWS Web控制台 - > VPC,您必须找到默认VPC ID和3个默认子网ID。 If your beanstalk uses custom VPC you must use these settings.
如果您的beanstalk使用自定义VPC,则必须使用这些设置。
.ebextensions/efs-volume.config
: .ebextensions/efs-volume.config
:
# efs-volume.config
# Copy this file to the .ebextensions folder in the root of your app source folder
option_settings:
aws:elasticbeanstalk:customoption:
EFSVolumeName: "EB-EFS-Volume"
VPCId: "vpc-xxxxxxxx"
SubnetUSWest2a: "subnet-xxxxxxxx"
SubnetUSWest2b: "subnet-xxxxxxxx"
SubnetUSWest2c: "subnet-xxxxxxxx"
Resources:
FileSystem:
Type: AWS::EFS::FileSystem
Properties:
FileSystemTags:
- Key: Name
Value:
Fn::GetOptionSetting: {OptionName: EFSVolumeName, DefaultValue: "EB_EFS_Volume"}
MountTargetSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Security group for mount target
SecurityGroupIngress:
- FromPort: '2049'
IpProtocol: tcp
SourceSecurityGroupId:
Fn::GetAtt: [AWSEBSecurityGroup, GroupId]
ToPort: '2049'
VpcId:
Fn::GetOptionSetting: {OptionName: VPCId}
MountTargetUSWest2a:
Type: AWS::EFS::MountTarget
Properties:
FileSystemId: {Ref: FileSystem}
SecurityGroups:
- {Ref: MountTargetSecurityGroup}
SubnetId:
Fn::GetOptionSetting: {OptionName: SubnetUSWest2a}
MountTargetUSWest2b:
Type: AWS::EFS::MountTarget
Properties:
FileSystemId: {Ref: FileSystem}
SecurityGroups:
- {Ref: MountTargetSecurityGroup}
SubnetId:
Fn::GetOptionSetting: {OptionName: SubnetUSWest2b}
MountTargetUSWest2c:
Type: AWS::EFS::MountTarget
Properties:
FileSystemId: {Ref: FileSystem}
SecurityGroups:
- {Ref: MountTargetSecurityGroup}
SubnetId:
Fn::GetOptionSetting: {OptionName: SubnetUSWest2c}
Resources: 资源:
AWS has instructions to automatically create and mount an EFS on elastic beanstalk. AWS具有在弹性beanstalk上自动创建和安装EFS的说明。 They can be found here
他们可以在这里找到
These instructions link to two config files to be customized and placed in .ebextensions folder of your deployment package. 这些说明链接到两个要自定义的配置文件,并放在部署包的.ebextensions文件夹中。
The file storage-efs-mountfilesystem.config needs to be further modified to work with Docker containers. 需要进一步修改文件storage-efs-mountfilesystem.config以使用Docker容器。 Add the following command:
添加以下命令:
02_restart:
command: "service docker restart"
And for multi-container environments Elastic Container Service has to be restarted as well (it was killed when docker was restarted above): 对于多容器环境,Elastic Container Service也必须重新启动(它在以上重新启动docker时被杀死):
03_start_eb:
command: |
start ecs
start eb-docker-events
sleep 120
test: sh -c "[ -f /etc/init/ecs.conf ]"
so the complete commands section of storage-efs-mountfilesystem.config is: 所以storage-efs-mountfilesystem.config的完整命令部分是:
commands:
01_mount:
command: "/tmp/mount-efs.sh"
02_restart:
command: "service docker restart"
03_start_eb:
command: |
start ecs
start eb-docker-events
sleep 120
test: sh -c "[ -f /etc/init/ecs.conf ]"
The reason this does not work "out-of-the-box" is because the docker daemon is started by the EC2 instance before the commands in .ebextensions are run. 这不起作用的原因是因为在运行.ebextensions中的命令之前,EC2实例启动了docker守护程序。 The startup order is:
启动顺序是:
At step one, the filesystem view the docker daemon provides to the containers is fixed. 在第一步,docker守护程序提供给容器的文件系统视图是固定的。 Therefore changes in the host filesystems made during step 3 are not reflected in the container's view.
因此,在步骤3中进行的主机文件系统的更改不会反映在容器的视图中。
One strange effect is that the container sees a mount point prior to the filesystem being mounted on the host. 一个奇怪的效果是容器在文件系统挂载到主机之前看到一个挂载点。 The host sees the mounted filesystem.
主机看到挂载的文件系统。 Therefore a file written by a container will be written to the host directory under the mounted directory, not the mounted filesystem.
因此,容器写入的文件将写入安装目录下的主机目录,而不是挂载的文件系统。 Unmounting the filesystem on the EC2 host will expose the container files written into the mount directory.
卸载EC2主机上的文件系统将公开写入mount目录的容器文件。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.