[英]AWS - Moving data from one S3 bucket to another with CloudFormation
I'm trying to create a stack with CloudFormation.我正在尝试使用 CloudFormation 创建一个堆栈。 The stack needs to take some data files from a central S3 bucket and copy them to it's own "local" bucket.
堆栈需要从中央 S3 存储桶中获取一些数据文件并将它们复制到它自己的“本地”存储桶。
I've written a lambda function to do this, and it works when I run it in the Lambda console with a test event (the test event uses the real central repository and successfully copies the file to a specified repo).我已经编写了一个 lambda 函数来执行此操作,当我在带有测试事件的 Lambda 控制台中运行它时它可以工作(测试事件使用真正的中央存储库并成功地将文件复制到指定的存储库)。
My current CloudFormation script does the following things:我当前的 CloudFormation 脚本执行以下操作:
It's at step 4 where it starts to go wrong - the Cloudformation execution seems to freeze here ( CREATE_IN_PROGESS
).它在第 4 步开始出错 - Cloudformation 执行似乎在这里冻结(
CREATE_IN_PROGESS
)。 Also, when I try to delete the stack, it seems to just get stuck on DELETE_IN_PROGRESS
instead.此外,当我尝试删除堆栈时,它似乎只是卡在
DELETE_IN_PROGRESS
。
Here's how I'm invoking the Lambda function in the CloudFormation script:下面是我在 CloudFormation 脚本中调用 Lambda 函数的方法:
"DataSync": {
"Type": "Custom::S3DataSync",
"Properties": {
"ServiceToken": { "Fn::GetAtt" : [ "S3DataSync", "Arn" ] },
"InputFile": "data/provided-as-ip-v6.json",
"OutputFile": "data/data.json"
}
},
"KeySync1": {
"Type": "Custom::S3DataSync",
"Properties": {
"ServiceToken": { "Fn::GetAtt" : [ "S3DataSync", "Arn" ] },
"InputFile": "keys/1/public_key.pem"
}
},
"KeySync2": {
"Type": "Custom::S3DataSync",
"Properties": {
"ServiceToken": { "Fn::GetAtt" : [ "S3DataSync", "Arn" ] },
"InputFile": "keys/2/public_key.pem"
}
}
And the Lambda function itself: Lambda 函数本身:
exports.handler = function(event, context) {
var buckets = {};
buckets.in = {
"Bucket":"central-data-repository",
"Key":"sandbox" + "/" + event.ResourceProperties.InputFile
};
buckets.out = {
"Bucket":"sandbox-data",
"Key":event.ResourceProperties.OutputFile || event.ResourceProperties.InputFile
};
var AWS = require('aws-sdk');
var S3 = new AWS.S3();
S3.getObject(buckets.in, function(err, data) {
if (err) {
console.log("Couldn't get file " + buckets.in.Key);
context.fail("Error getting file: " + err)
}
else {
buckets.out.Body = data.Body;
S3.putObject(buckets.out, function(err, data) {
if (err) {
console.log("Couln't write to S3 bucket " + buckets.out.Bucket);
context.fail("Error writing file: " + err);
}
else {
console.log("Successfully copied " + buckets.in.Key + " to " + buckets.out.Bucket + " at " + buckets.out.Key);
context.succeed();
}
});
}
});
}
Your Custom Resource function needs to send signals back to CloudFormation to indicate completion, status, and any returned values. 您的自定义资源功能需要将信号发送回CloudFormation以指示完成,状态和任何返回的值。 You will see CREATE_IN_PROGRESS as the status in CloudFormation until you notify it that your function is complete.
您将看到CREATE_IN_PROGRESS作为CloudFormation中的状态,直到您通知它您的功能已完成。
The generic way of signaling CloudFormation is to post a response to a pre-signed S3 URL. 信令CloudFormation的通用方式是发布对预先签名的S3 URL的响应。 But there is a cfn-response module to make this easier in Lambda functions.
但是有一个cfn-response模块可以使Lambda函数更容易实现。 Interestingly, the two examples provided for Lambda-backed Custom Resources use different methods:
有趣的是,为Lambda支持的自定义资源提供的两个示例使用不同的方法:
Yup, i did the same thing.是的,我做了同样的事情。 We need to upload(PUT request) the status of our request.(Need to send the status as SUCCESS)
我们需要上传(PUT 请求)我们请求的状态。(需要发送状态为 SUCCESS)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.