Cloudfront 在没有公共访问的情况下为 S3 存储桶源提供通过 AWS CDK Python 创建的访问被拒绝的响应

Question

使用 AWS CDK 为 S3 存储桶创建 Cloud Front Web 分发，无需公共访问。 能够创建 Origin 访问身份并进行部署，但在成功部署后，我在浏览器上收到访问被拒绝的响应。

Grant Read Permissions on Bucket from Origin 设置将设置为 No，手动将其设置为 Yes 一切正常，但此设置需要通过 AWS CDK 和 python 实现。 下面是我的代码。

from aws_cdk import aws_cloudfront as front, aws_s3 as s3

class CloudFrontStack(core.Stack):        
    def __init__(self, scope: core.Construct, idx: str, **kwargs) -> None:
        super().__init__(scope, idx, **kwargs)

        bucket = s3.Bucket.from_bucket_name(self, 'CloudFront',bucket_name="bucket_name")

        oia = aws_cloudfront.OriginAccessIdentity(self, 'OIA', comment="Created By CDK")
        bucket.grant_read(oia)

        s3_origin_source = aws_cloudfront.S3OriginConfig(s3_bucket_source=bucket, origin_access_identity=oia)

        source_config = aws_cloudfront.SourceConfiguration(s3_origin_source=s3_origin_source,
                                                           origin_path="bucket_path",
                                                           behaviors=[aws_cloudfront.Behavior(is_default_behavior=True)])

        aws_cloudfront.CloudFrontWebDistribution(self, "cloud_front_name",
                                                 origin_configs=[source_config],
                                                 comment='Cloud Formation created',
                                                 default_root_object='index.html')

我也尝试将权限添加到下面但仍然没有运气。

policyStatement = aws_iam.PolicyStatement()
policyStatement.add_resources()

policyStatement.add_actions('s3:GetBucket*');
policyStatement.add_actions('s3:GetObject*');
policyStatement.add_actions('s3:List*');
policyStatement.add_resources(bucket.bucket_arn);
policyStatement.add_canonical_user_principal(oia.cloud_front_origin_access_identity_s3_canonical_user_id);
code_bucket.add_to_resource_policy(policyStatement);

Answer 1

我试图模仿这一点，并成功地将 Cloudfront 发行版集成到私有 S3 存储桶中。 但是，我在我的堆栈中使用了 TS。 我相信将下面的代码与 Python 版本相关联会很容易。 假设dist有一个index.html文件

aws-cdk v1.31.0（2020 年 3 月 29 日最新）

import { App, Stack, StackProps } from '@aws-cdk/core';
import { BucketDeployment, Source } from '@aws-cdk/aws-s3-deployment';
import { CloudFrontWebDistribution, OriginAccessIdentity } from '@aws-cdk/aws-cloudfront';
import { BlockPublicAccess, Bucket, BucketEncryption } from '@aws-cdk/aws-s3';

export class HelloCdkStack extends Stack {
  constructor(scope: App, id: string, props?: StackProps) {
    super(scope, id, props);

    const myFirstBucket = new Bucket(this, 'MyFirstBucket', {
      versioned: true,
      encryption: BucketEncryption.S3_MANAGED,
      bucketName: 'cdk-example-bucket-for-test',
      websiteIndexDocument: 'index.html',
      blockPublicAccess: BlockPublicAccess.BLOCK_ALL
    });

    new BucketDeployment(this, 'DeployWebsite', {
      sources: [Source.asset('dist')],
      destinationBucket: myFirstBucket
    });

    const oia = new OriginAccessIdentity(this, 'OIA', {
      comment: "Created by CDK"
    });
    myFirstBucket.grantRead(oia);

    new CloudFrontWebDistribution(this, 'cdk-example-distribution', {
      originConfigs: [
        {
          s3OriginSource: {
            s3BucketSource: myFirstBucket,
            originAccessIdentity: oia
          },
          behaviors: [
            { isDefaultBehavior: true }
          ]
        }
      ]
    });
  }
}

==更新== [没有虚拟主机的 S3 存储桶]

这是一个示例，其中 S3 用作没有 Web 托管的 Origin。 它按预期工作。

import { App, Stack, StackProps } from '@aws-cdk/core';
import { BucketDeployment, Source } from '@aws-cdk/aws-s3-deployment';
import { CloudFrontWebDistribution, OriginAccessIdentity } from '@aws-cdk/aws-cloudfront';
import { BlockPublicAccess, Bucket, BucketEncryption } from '@aws-cdk/aws-s3';

export class CloudfrontS3Stack extends Stack {
  constructor(scope: App, id: string, props?: StackProps) {
    super(scope, id, props);

    // Create bucket (which is not a static website host), encrypted AES-256 and block all public access
    // Only Cloudfront access to S3 bucket
    const testBucket = new Bucket(this, 'TestS3Bucket', {
      encryption: BucketEncryption.S3_MANAGED,
      bucketName: 'cdk-static-asset-dmahapatro',
      blockPublicAccess: BlockPublicAccess.BLOCK_ALL
    });

    // Create Origin Access Identity to be use Canonical User Id in S3 bucket policy
    const originAccessIdentity = new OriginAccessIdentity(this, 'OAI', {
      comment: "Created_by_dmahapatro"
    });
    testBucket.grantRead(originAccessIdentity);

    // Create Cloudfront distribution with S3 as Origin
    const distribution = new CloudFrontWebDistribution(this, 'cdk-example-distribution', {
      originConfigs: [
        {
          s3OriginSource: {
            s3BucketSource: testBucket,
            originAccessIdentity: originAccessIdentity
          },
          behaviors: [
            { isDefaultBehavior: true }
          ]
        }
      ]
    });

    // Upload items in bucket and provide distribution to create invalidations
    new BucketDeployment(this, 'DeployWebsite', {
      sources: [Source.asset('dist')],
      destinationBucket: testBucket,
      distribution,
      distributionPaths: ['/images/*.png']
    });
  }
}

== UPDATE == [S3 Bucket 导入而不是在同一个堆栈中创建]

当我们引用现有的 S3 存储桶时，可以重新创建该问题。

原因：
问题的根本原因就在这行代码中。 对于导入的 S3 存储桶， autoCreatePolicy 将始终为false 。 要使addResourcePolicy起作用，导入的存储桶必须已经有一个现有的存储桶策略，以便可以附加新的策略语句，或者手动创建新的 BucketPolicy 并添加策略语句。 在下面的代码中，我手动创建了存储桶策略并添加了所需的策略语句。 这与github 问题 #941非常接近，但细微的区别在于在堆栈中创建存储桶与导入已创建的存储桶。

import { App, Stack, StackProps } from '@aws-cdk/core';
import { CloudFrontWebDistribution, OriginAccessIdentity } from '@aws-cdk/aws-cloudfront';
import { Bucket, BucketPolicy } from '@aws-cdk/aws-s3';
import { PolicyStatement } from '@aws-cdk/aws-iam';

export class CloudfrontS3Stack extends Stack {
  constructor(scope: App, id: string, props?: StackProps) {
    super(scope, id, props);

    const testBucket = Bucket.fromBucketName(this, 'TestBucket', 'dmahapatro-personal-bucket');

    // Create Origin Access Identity to be use Canonical User Id in S3 bucket policy
    const originAccessIdentity = new OriginAccessIdentity(this, 'OAI', {
      comment: "Created_by_dmahapatro"
    });

    // This does not seem to work if Bucket.fromBucketName is used
    // It works for S3 buckets which are created as part of this stack
    // testBucket.grantRead(originAccessIdentity);

    // Explicitly add Bucket Policy 
    const policyStatement = new PolicyStatement();
    policyStatement.addActions('s3:GetBucket*');
    policyStatement.addActions('s3:GetObject*');
    policyStatement.addActions('s3:List*');
    policyStatement.addResources(testBucket.bucketArn);
    policyStatement.addResources(`${testBucket.bucketArn}/*`);
    policyStatement.addCanonicalUserPrincipal(originAccessIdentity.cloudFrontOriginAccessIdentityS3CanonicalUserId);

    // testBucket.addToResourcePolicy(policyStatement);

    // Manually create or update bucket policy
    if( !testBucket.policy ) {
      new BucketPolicy(this, 'Policy', { bucket: testBucket }).document.addStatements(policyStatement);
    } else {
      testBucket.policy.document.addStatements(policyStatement);
    }

    // Create Cloudfront distribution with S3 as Origin
    const distribution = new CloudFrontWebDistribution(this, 'cdk-example-distribution', {
      originConfigs: [
        {
          s3OriginSource: {
            s3BucketSource: testBucket,
            originAccessIdentity: originAccessIdentity
          },
          behaviors: [
            { isDefaultBehavior: true }
          ]
        }
      ]
    });
  }
}

Answer 2

添加到@dmahapatro 答案中， Distribution界面再次发生了变化。

截至 2022 年 8 月，CDK 版本 2.37.1， Distribution现在应该采用BehaviorOptions类型的defaultBehavior键。

originAccessIdentity现在被添加到S3Origin的S3Origin中。

前

new CloudFrontWebDistribution(this, 'cdk-example-distribution', {
  originConfigs: [
    {
      s3OriginSource: {
        s3BucketSource: myFirstBucket,
        originAccessIdentity: oia
      },
      behaviors: [
        { isDefaultBehavior: true }
      ]
    }
  ]
});

后

const distribution = new Distribution(this, "cdk-example-distribution", {
  defaultBehavior: {
    origin: new S3Origin(testBucket, {
      originAccessIdentity: originAccessIdentity,
    }),
  }
});

Answer 3

作为对上述答案的补充，我将发布一个有效的 Python 解决方案，该解决方案允许 Cloudfront 访问私有 S3 存储桶。 要做到这一点，三点必不可少：

1. 创建 Cloudfront 源访问身份：

    origin_access_identity = cloudfront.OriginAccessIdentity( self, 'cdkTestOriginAccessIdentity' )

2. 授予对Cloudfront Origin Access Identity 的访问权限以从 S3 存储桶中读取： self.s3_bucket.grant_read(origin_access_identity)尚未实施）

3. 必须将 Origin Access Identity 实例提供给 Cloudfront Distribution 实例，以允许 S3 允许 Cloudfront 请求：

    origin=origins.S3Origin( self.s3_bucket, origin_access_identity=access_identity)

最终代码如下：

   import aws_cdk as cdk
   from aws_cdk import (
     aws_s3 as s3,
     aws_s3_deployment as s3deploy,
     aws_cloudfront as cloudfront,
     aws_cloudfront_origins as origins
   )


    class Presentation(cdk.Stack):
      def __init__(self, scope, id, **kwargs):
        super().__init__(scope, id, **kwargs)

        # Define a S3 bucket that contains a static website
        self.s3_bucket = self._create_hosting_s3_bucket()

        # Creates Cloudfront Origin Access Identity to access a restricted S3
        origin_access_identity = cloudfront.OriginAccessIdentity(
            self,
            'cdkTestOriginAccessIdentity'
        )

        # Allows Origin Access Control to read from S3
        self.s3_bucket.grant_read(origin_access_identity)

        # Define Cloudfront CDN that delivers from S3 bucket
        self.cdn = self._create_cdn(access_identity=origin_access_identity)

      def _create_hosting_s3_bucket(self):
        """ Returns a S3 instance that serves a static website """
        website_bucket = s3.Bucket(
            self,
            'static-website-for-cdkdemo',
            removal_policy=cdk.RemovalPolicy.DESTROY,
            access_control=s3.BucketAccessControl.PRIVATE,
        )

        # Populates bucket with files from local file system
        s3deploy.BucketDeployment(
            self,
            'DeployWebsite',
            destination_bucket=website_bucket,
            sources=[
                s3deploy.Source.asset('../frontend')
            ],
            retain_on_delete=False,
        )
        return website_bucket

      def _create_cdn(self, access_identity):
        """ Returns a CDN that delivers from a S3 bucket """
        return cloudfront.Distribution(
            self,
            'myDist',
            default_behavior=cloudfront.BehaviorOptions(
                origin=origins.S3Origin(
                    self.s3_bucket,
                    origin_access_identity=access_identity,
                )
            )
        )