not able to download bootstrap file from S3 when using in Cloudformation

Question

we are trying to deploy a EC2 stack which has LaunchConfig, AutoScaling group and IAM role. in Launch config we have configured to execute the bash script with help of "AWS::CloudFormation::Init". when EC2 machines get launched in its not able to fetch the bootstrap file from S3. below is the error logs. cfn-init.log

{
2020-02-21 17:48:48,663 https://forums.aws.amazon.com/ HTTP Error 404 : <?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>404 - Not Found</title>
</head>
<body>
<h1>404 - Not Found</h1>
</body>
</html>
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/cfnbootstrap/util.py", line 162, in _retry
return f(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/cfnbootstrap/util.py", line 286, in get_role_creds
resp.raise_for_status()
File "/usr/lib/python2.7/dist-packages/cfnbootstrap/packages/requests/models.py", line 834, in raise_for_status
raise HTTPError(http_error_msg, response=self)
HTTPError: 404 Client Error: Not Found
}

cloud-init-output.log

{
Error occurred during build: Failed to retrieve https://BUCKET_NAME.s3.amazonaws.com/BUCKET_PREFIX/scripts/bastion_bootstrap.sh: https://forums.aws.amazon.com/ HTTP Error 404 : <?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>404 - Not Found</title>
</head>
<body>
<h1>404 - Not Found</h1>
</body>
</html>
}

and below is the CF template which we have:

  BastionLaunchConfiguration:
    Type: 'AWS::AutoScaling::LaunchConfiguration'
    Metadata:
      'AWS::CloudFormation::Authentication':
        S3AccessCreds:
          type: S3
          roleName: !Ref BastionHostProfile
          buckets:
            - !Ref QSS3BucketName
      'AWS::CloudFormation::Init':
        config:
          files:
            /tmp/bastion_bootstrap.sh:
              source: !Sub 'https://${QSS3BucketName}.s3.${QSS3Region}.amazonaws.com/${QSS3KeyPrefix}scripts/bastion_bootstrap.sh'
              mode: '000550'
              owner: root
              group: root
              authentication: S3AccessCreds
          commands:
            b-bootstrap:
              cwd: '/tmp/'
              command: !Join
                - ''
                - - ./bastion_bootstrap.sh
                  - ' --banner '
                  - !Ref BastionBanner
                  - ' --enable '
                  - !Ref EnableBanner
                  - ' --tcp-forwarding '
                  - !Ref EnableTCPForwarding
                  - ' --x11-forwarding '
                  - !Ref EnableX11Forwarding
    Properties:
      AssociatePublicIpAddress: true
      PlacementTenancy: !Ref BastionTenancy
      KeyName: !Ref KeyPairName
      IamInstanceProfile: !Ref BastionHostProfile
      ImageId: !If
        - UseOSImageOverride
        - !Ref OSImageOverride
        - !FindInMap
          - AWSAMIRegionMap
          - !Ref 'AWS::Region'
          - !FindInMap
            - LinuxAMINameMap
            - !Ref BastionAMIOS
            - Code
      SecurityGroups:
        - !Ref BastionSecurityGroup
      InstanceType: !Ref BastionInstanceType
      BlockDeviceMappings:
        - DeviceName: /dev/xvda
          Ebs:
            VolumeSize: !Ref RootVolumeSize
            VolumeType: gp2
            Encrypted: true
            DeleteOnTermination: true

Below is the IAM role resource which we are assigning to launch config:

  BastionHostRole:
    Condition: CreateIAMRole
    Type: 'AWS::IAM::Role'
    Properties:
      Path: / 
      AssumeRolePolicyDocument:
        Statement:
          - Action:
              - 'sts:AssumeRole'
            Principal:
              Service:
                - ec2.amazonaws.com
            Effect: Allow
        Version: 2012-10-17
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/service-role/AmazonEC2RoleforSSM'
  BastionHostPolicy:
    Type: 'AWS::IAM::Policy'
    Properties:
      PolicyName: BastionPolicy
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Action:
              - 's3:*'
            Resource: !Sub
              - 'arn:${Partition}:s3:::${QSS3BucketName}/${QSS3KeyPrefix}*'
              - Partition: !If
                  - GovCloudCondition
                  - aws-us-gov
                  - aws
            Effect: Allow
          - Action:
              - 'logs:CreateLogStream'
              - 'logs:GetLogEvents'
              - 'logs:PutLogEvents'
              - 'logs:DescribeLogGroups'
              - 'logs:DescribeLogStreams'
              - 'logs:PutRetentionPolicy'
              - 'logs:PutMetricFilter'
              - 'logs:CreateLogGroup'
            Resource: !Sub
              - arn:${Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:${BastionMainLogGroup}:*
              - Partition: !If
                  - GovCloudCondition
                  - aws-us-gov
                  - aws
            Effect: Allow
          - Action:
              - 'ec2:AssociateAddress'
              - 'ec2:DescribeAddresses'
            Resource: '*'
            Effect: Allow
      Roles:
        - !If
          - CreateIAMRole
          - !Ref BastionHostRole
          - !Ref AlternativeIAMRole
  BastionHostProfile:
    DependsOn: BastionHostPolicy
    Type: 'AWS::IAM::InstanceProfile'
    Properties:
      Roles:
        - !If
          - CreateIAMRole
          - !Ref BastionHostRole
          - !Ref AlternativeIAMRole
      Path: /

any suggestion, how we can fix this?

Answer 1

At the very first look it looks like you have a typo here.

'https://${QSS3BucketName}.s3.${QSS3Region}.amazonaws.com/${QSS3KeyPrefix}scripts/bastion_bootstrap.sh'

The usual format is <BucketName>.s3-<Region> where as you have a period between s3 and region.

not able to download bootstrap file from S3 when using in Cloudformation

Question

1 answers

solution1
0 2020-02-24 13:58:11

not able to download bootstrap file from S3 when using in Cloudformation

Question

1 answers

solution1 0 2020-02-24 13:58:11

solution1
0 2020-02-24 13:58:11