将 EBS 卷（不是快照）挂载到 Elastic Beanstalk EC2

Question

我正在将遗留应用程序迁移到 Elastic Beanstalk。 它需要持久存储（暂时）。 我想挂载 EBS 卷。

我希望以下内容可以在.ebextensions/ebs.config中使用：

commands:
  01mkdir:
    command: "mkdir /data"
  02mount:
    command: "mount /dev/sdh /data"

option_settings:
  - namespace: aws:autoscaling:launchconfiguration
    option_name: BlockDeviceMappings
    value: /dev/sdh=vol-XXXXX

https://blogs.aws.amazon.com/application-management/post/Tx224DU59IG3OR9/Customize-Ephemeral-and-EBS-Volumes-in-Elastic-Beanstalk-Environments

但不幸的是，我收到以下错误消息“参数 snapshotId 的 (vol-XXXX) 无效。应为：'snap-...'。”

很明显，这种方法只允许快照。 任何人都可以建议修复或替代方法。

Answer 1

我找到了解决方案。 可以通过删除“sleep 10”来改进它，但不幸的是，因为aws ec2 attach-volume是异步的并且在附件发生之前立即返回。

container_commands:
  01mount:
    command: "aws ec2 attach-volume --volume-id vol-XXXXXX --instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id) --device /dev/sdh"
    ignoreErrors: true
  02wait:
    command: "sleep 10"
  03mkdir:
    command: "mkdir /data"
    test: "[ ! -d /data ]"
  04mount:
    command: "mount /dev/sdh /data"
    test: "! mountpoint -q /dev/sdh"

注意。 理想情况下，它将在commands部分运行而不是container_commands但环境变量未及时设置。

Answer 2

要添加到@ Simon的答案（以避免陷入困境的陷阱）：

• 如果正在安装的持久存储最终将在Docker容器中使用（例如，如果您正在运行Jenkins并希望保留jenkins_home），则需要在运行mount之后重新启动docker容器。
• 您需要对EB假设角色策略中的EC2实例（或实例/ * ARN）和要附加的卷（或卷/ * ARN）允许“ec2：AttachVolumes”操作。 如果没有这个， aws ec2 attach-volume命令将失败。
• 您还需要将--region传递给aws ec2 ...命令（至少在撰写本文时）

Answer 3

或者，您可以考虑使用弹性文件系统（EFS）存储，而不是使用EBS卷。 AWS已发布了一个脚本，介绍如何将EFS卷安装到Elastic Beanstalk EC2实例，并且它还可以同时附加到多个EC2实例（这对于EBS是不可能的）。

http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/services-efs.html

Answer 4

这是一个可以放入.ebextensions的配置文件。 您需要提供要附加的VOLUME_ID 。 测试命令使得只能根据需要进行连接和安装，以便您可以重复进行eb deploy而不会出现错误。

container_commands:
  00attach:
    command: |
      export REGION=$(/opt/aws/bin/ec2-metadata -z | awk '{print substr($2, 0, length($2)-1)}')
      export INSTANCE_ID=$(/opt/aws/bin/ec2-metadata -i | awk '{print $2}')
      export VOLUME_ID=$(aws ec2 describe-volumes --region ${REGION} --output text --filters Name=tag:Name,Values=tf-trading-prod --query 'Volumes[*].VolumeId')

      aws ec2 attach-volume --region ${REGION} --device /dev/sdh --instance-id ${INSTANCE_ID} --volume-id ${VOLUME_ID}
      aws ec2 wait volume-in-use --region ${REGION} --volume-ids ${VOLUME_ID}
      sleep 1
    test: "! file -E /dev/xvdh"
  01mkfs:
    command: "mkfs -t ext3 /dev/xvdh"
    test: "file -s /dev/xvdh | awk '{print $2}' | grep -q data"
  02mkdir:
    command: "mkdir -p /data"
  03mount:
    command: "mount /dev/xvdh /data"
    test: "! mountpoint /data"

Answer 5

必须使用container_commands因为当commands运行时源包还没有完全解压。

.ebextensions/whatever.config

container_commands:
  chmod:
    command: chmod +x .platform/hooks/predeploy/mount-volume.sh

预部署挂钩在容器命令之后但在部署之前运行。 无需重新启动您的 docker 容器，即使它在附加的 ebs 卷上安装了一个目录，因为 beanstalk 在预部署挂钩完成后将其旋转起来。 您可以在日志中看到它。

.platform/hooks/predeploy/mount-volume.sh

#!/bin/sh

# Make sure LF line endings are used in the file, otherwise there would be an error saying "file not found".

# All platform hooks run as root user, no need for sudo.

# Before attaching the volume find out the root volume's name, so that we can later use it for filtering purposes.
# -d – to filter out partitions.
# -P – to display the result as key-value pairs.
# -o – to output only the matching part.
# lsblk strips the "/dev/" part
ROOT_VOLUME_NAME=$(lsblk -d -P | grep -o 'NAME="[a-z0-9]*"' | grep -o '[a-z0-9]*')

aws ec2 attach-volume --volume-id vol-xxx --instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id) --device /dev/sdf --region us-east-1
# The above command is async, so we need to wait.
aws ec2 wait volume-in-use --volume-ids vol-xxx --region us-east-1

# Now lsblk should show two devices. We figure out which one is non-root by filtering out the stored root volume name.
NON_ROOT_VOLUME_NAME=$(lsblk -d -P | grep -o 'NAME="[a-z0-9]*"' | grep -o '[a-z0-9]*' | awk -v name="$ROOT_VOLUME_NAME" '$0 !~ name')

FILE_COMMAND_OUTPUT=$(file -s /dev/$NON_ROOT_VOLUME_NAME)

# Create a file system on the non-root device only if there isn't one already, so that we don't accidentally override it.
if test "$FILE_COMMAND_OUTPUT" = "/dev/$NON_ROOT_VOLUME_NAME: data"; then
  mkfs -t xfs /dev/$NON_ROOT_VOLUME_NAME
fi

mkdir /data

mount /dev/$NON_ROOT_VOLUME_NAME /data

# Need to make sure that the volume gets mounted after every reboot, because by default only root volume is automatically mounted.

cp /etc/fstab /etc/fstab.orig

NON_ROOT_VOLUME_UUID=$(lsblk -d -P -o +UUID | awk -v name="$NON_ROOT_VOLUME_NAME" '$0 ~ name' | grep -o 'UUID="[-0-9a-z]*"' | grep -o '[-0-9a-z]*')

# We specify 0 to prevent the file system from being dumped, and 2 to indicate that it is a non-root device.
# If you ever boot your instance without this volume attached, the nofail mount option enables the instance to boot
# even if there are errors mounting the volume.
# Debian derivatives, including Ubuntu versions earlier than 16.04, must also add the nobootwait mount option.
echo "UUID=$NON_ROOT_VOLUME_UUID /data xfs defaults,nofail 0 2" | tee -a /etc/fstab

很确定我用grep和awk做的事情可以用更简洁的方式完成。 我不太擅长 Linux。

实例配置文件应包括以下权限：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:AttachVolume",
                "ec2:DetachVolume",
                "ec2:DescribeVolumes"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:volume/*",
                "arn:aws:ec2:*:*:instance/*"
            ]
        }
    ]
}

您必须确保将 ebs 卷部署在与 beantalk 相同的 AZ 中，并且您使用 SingleInstance 部署。 然后，如果您的实例崩溃，ASG 将终止它，创建另一个实例，并将卷附加到新实例以保留所有数据。

Answer 6

这是缺少配置：

commands:
  01mount:
    command: "export AWS_ACCESS_KEY_ID=<replace by your AWS key> && export AWS_SECRET_ACCESS_KEY=<replace by your AWS secret> && aws ec2 attach-volume --volume-id <replace by you volume id> --instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id) --device /dev/xvdf --region <replace with your region>"
    ignoreErrors: true
  02wait:
    command: "sleep 10"
  03mkdir:
    command: "mkdir /home/lucene"
    test: "[ ! -d /home/lucene ]"
  04mount:
    command: "mount /dev/xvdf /home/lucene"
    test: "! mountpoint -q /dev/xvdf"