Cryptic CloudFormation failure when creating CloudFront Distribution

Question

. 。 Answers to this question are eligible for a +200 reputation bounty. JSON Brody wants to to this question. 这个问题。

I have a CloudFormation template set up to track a CloudFront distribution among other things. Getting this set up, I created an AWS::CertificateManager::Certificate and an AWS::CloudFront::Distribution resource, where the CDN just serves from a non-website S3 origin.

When I run the change set, I get this incredibly vague failure. "Access denied for operation 'AWS::CloudFront::Distribution'." kind of loses me here. For one thing, it's not clear to me what operation this is supposed to be. On top of that, the stack rollback after this is incomplete. The CloudFormation events don't even show an attempt to remove the CDN or the cert, and when I try to hit the CloudFront URL from my browser, it works flawlessly, so I am not even sure what my template was trying to do here that failed. In fact, the only reason this is an issue for me is because the incomplete rollback tries to revert my lambdas in the stack to nodejs8.10, which causes larger failures. If that weren't an issue, I don't know that I would feel the effects of this vague error.

Template, based on the static site sample from a couple of years ago:

AWSTemplateFormatVersion: 2010-09-09
Transform:
- AWS::Serverless-2016-10-31
- AWS::CodeStar

Parameters:
  ProjectId:
    Type: String
    Description: AWS CodeStar projectID used to associate new resources to team members
  CodeDeployRole:
    Type: String
    Description: IAM role to allow AWS CodeDeploy to manage deployment of AWS Lambda functions
  Stage:
    Type: String
    Description: The name for a project pipeline stage, such as Staging or Prod, for which resources are provisioned and deployed.
    Default: ''

Globals:
  Api:
    BinaryMediaTypes:
      - image~1png
  Function:
    Runtime: nodejs14.x
    AutoPublishAlias: live
    DeploymentPreference:
      Enabled: true
      Type: Canary10Percent5Minutes
      Role: !Ref CodeDeployRole

Resources:

  MahCert:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: domain.com
      DomainValidationOptions:
        - DomainName: domain.com
          HostedZoneId: Z2GZX5ZQI1HO5L
      SubjectAlternativeNames:
        - '*.domain.com'
      CertificateTransparencyLoggingPreference: ENABLED
      ValidationMethod: DNS

  CloudFrontCDN:
    Type: AWS::CloudFront::Distribution
    Properties:
      DistributionConfig:
        Comment: Source for all static resources
        PriceClass: PriceClass_100
        Aliases:
          - domain.com
        ViewerCertificate:
          AcmCertificateArn: !Ref MahCert
          MinimumProtocolVersion: TLSv1.2_2021
          SslSupportMethod: sni-only
        DefaultRootObject: index.html
        DefaultCacheBehavior:
          ViewerProtocolPolicy: redirect-to-https
          CachePolicyId: b2884449-e4de-46a7-ac36-70bc7f1ddd6d
          TargetOriginId: SiteBucket
        Enabled: True
        Origins:
          - DomainName: <my_bucket>.s3.amazonaws.com
            Id: SiteBucket
            S3OriginConfig:
              OriginAccessIdentity: ''

  ServerlessRestApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: Prod
      DefinitionBody:
        swagger: 2.0
        info:
          title: Static resource proxy

        paths:
          /static/{proxy+}:
            get:
              x-amazon-apigateway-integration:
                httpMethod: ANY
                type: http_proxy
                uri: <my_bucket>.s3.amazonaws.com/static/{proxy}
              responses: {}

  GetHelloWorld:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.get
      Role:
        Fn::GetAtt:
        - LambdaExecutionRole
        - Arn
      Events:
        GetEvent:
          Type: Api
          Properties:
            Path: /
            Method: get
        ProxyEvent:
          Type: Api
          Properties:
            Path: /{proxy+}
            Method: any
            
  GetStaticContent:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.getResource
      Role:
        Fn::GetAtt:
        - LambdaExecutionRole
        - Arn
      Events:
        GetResourceEvent:
          Type: Api
          Properties:
            Path: /static/{folder}/{file}
            Method: get
            
  GetQuote:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.getQuote
      Role:
        Fn::GetAtt:
        - LambdaDynamoDBReadRole
        - Arn
      Events:
        GetRandomQuoteEvent:
          Type: Api
          Properties:
            Path: /getquote
            Method: get
        GetQuoteEvent:
          Type: Api
          Properties:
            Path: /getquote/{id}
            Method: get
            
  LambdaExecutionRole:
    Description: Creating service role in IAM for AWS Lambda
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub 'CodeStar-${ProjectId}-Execution${Stage}'
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service: [lambda.amazonaws.com]
          Action: sts:AssumeRole
      Path: /
      ManagedPolicyArns:
        - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'
      PermissionsBoundary: !Sub 'arn:${AWS::Partition}:iam::${AWS::AccountId}:policy/CodeStar_${ProjectId}_PermissionsBoundary'
      
  LambdaDynamoDBReadRole:
    Description: Creating service role in IAM for AWS Lambda
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub '${ProjectId}-DynamoDB-Read'
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service: [lambda.amazonaws.com]
          Action: sts:AssumeRole
      Path: /
      Policies:
        -
          PolicyName: "dynamodb-read-quotes"
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              -
                Effect: "Allow"
                Action:
                  - "dynamodb:GetItem"
                  - "dynamodb:DescribeTable"
                Resource: "<dynamo_arn>"
      ManagedPolicyArns:
        - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'

Note - domain.com is not the actual domain I'm using here.

Updates:

I deleted the stack completely and recreated it from this template, thinking something was wrong with the history of the stack. However, I got the same error.

The IAM role that the stack uses has these permissions

Indeed, the problem persisted even after granting this role full write access to CloudFront resources.

Answer 1

Based on the chat discussion.

The cause of the issue was found to be missing IAM permissions for the IAM role that is used to deploy the stack. Specifically, the permission that was missing was:

• cloudfront:GetDistribution - Grants permission to get the information about a web distribution

Adding that permission to the role, solved the problem.

To find the missing permission, CloudTrial's Event History was used.