AWS Boto3-等待直到将卷附加到EC2实例

Question

What I need to achieve is to have Lambda function that creates EBS volume, attaches it to EC2 instance, formats it and mounts it under /data directory.

I am using ssm:RunCommand ( client.send_command ) to execute shell script that is supposed to format and mount the volume but the code is failing because volume is not yet attached to the instance when I am calling RunCommand.

I am using EC2.Waiter.VolumeInUse to wait till the volume is attached but it seems that it is not working correctly.

Here is my code

import boto3

# HARDCODED VALUES FOR TESTING
AVAILABILITY_ZONE = 'us-east-1d'
INSTANCE_ID = 'i-0bd640b495fd7d77c'

ec2_client = boto3.client('ec2')
ssm_client = boto3.client('ssm')

volume_available_waiter = ec2_client.get_waiter('volume_available')
volume_attached_waiter = ec2_client.get_waiter('volume_in_use')


def lambda_handler(event, context):
    try:
        # create 8 GB general purpose volume in given AZ
        create_volume_response = ec2_client.create_volume(
            AvailabilityZone=AVAILABILITY_ZONE,
            Size=8,
            VolumeType='gp2'
        )

        # retrieve volume id and wait till it is available
        volume_id = create_volume_response['VolumeId']
        volume_available_waiter.wait(
            VolumeIds=[volume_id]
        )

        # attach newly created volume to a given instance
        ec2_client.attach_volume(
            Device='/dev/xvdh',
            InstanceId=INSTANCE_ID,
            VolumeId=volume_id
        )

        # wait till the volume is properly attached to EC2 instance
        volume_attached_waiter.wait(
            VolumeIds=[volume_id]
        )

        # use SSM RunCommand to format and mount volume
        ssm_client.send_command(
            InstanceIds=[INSTANCE_ID],
            DocumentName='AWS-RunShellScript',
            Parameters={
                'commands': [
                    'echo "STARTING MOUNT SEQUENCE"'
                    'echo $(lsblk)'
                    'mkfs -t xfs /dev/xvdh',
                    'mkdir /data',
                    'mount /dev/xvdh /data'
                ]
            }
        )

    except Exception as e:
        print(e)

    return 0

When checking logs cat /var/log/messages , I can clearly see that new volume is not attached yet from output of echo $(lsblk) .

What is the proper way of waiting until the volume is attached to EC2 instance?

Answer 1

In this case, the proper way would be to wait until the volume is attached in SSM, not by letting your lambda hanging and waiting.
Since you are already using SSM, you will need to make an SSM Automation document that would wait for the volume to be attached and then execute your RunCommand to format and mount the volume.
Your Document would need to add 2 steps:
1- aws:waitForAwsResourceProperty to wait until the volume is attached
2- aws:runCommand to execute your shell script

First, create your SSM automation document:

---
description: "Automation Document Example YAML Template"
schemaVersion: "0.3"
assumeRole: "{{ AutomationAssumeRole }}"
parameters:
  InstanceId:
    type: "String"
    description: "(Required) The ID of the EC2 Instance."
  VolumeId:
    type: "String"
    description: "(Required) The ID of the volume."
  AutomationAssumeRole:
    type: "String"
    description: "(Optional) The ARN of the role that allows Automation to perform the actions on your behalf."
    default: ""

mainSteps:
- name: "VerifyVolumeAttached"
  action: "aws:waitForAwsResourceProperty"
  timeoutSeconds: 600
  inputs:
    Service: "ec2"
    Api: "DescribeVolumes"
    VolumeIds: ["{{ VolumeId }}"]
    PropertySelector: "$.Volumes[0].Attachments[0].State"
    DesiredValues:
    - "attached"

- name: "MountVolume"
  action: "aws:runCommand"
  inputs:
    DocumentName: "AWS-RunShellScript"
    InstanceIds:
    - "{{InstanceId}}"
    Parameters:
      commands: ['echo "STARTING MOUNT SEQUENCE"','echo $(lsblk)','mkfs -t xfs /dev/xvdh','mkdir /data','mount /dev/xvdh /data']

Then you will need to create an IAM Role for SSM to with the required permissions to Runcommand and DescribeVolumes.
Then replace the send command block in your lambda to:

# Start SSM automation execution    
ssm_client.start_automation_execution(DocumentName=your_automation_document_name,Parameters={"InstanceId": [INSTANCE_ID],"VolumeId":[volume_id],"AutomationAssumeRole":[ssm_automation_role_arn]}