[英]Unable to trigger AWS Lambda by upload to AWS S3
I am trying to build a Kibana dashboard fed with twitter data collected via AWS Kinesis firehose where data passes into an S3 bucket which triggers a Lambda function which passes the data to AWS Elastic Search and then to Kibana. 我正在尝试构建一个Kibana仪表板,该仪表板填充有通过AWS Kinesis firehose收集的twitter数据,其中数据传递到S3存储桶中,该S3存储桶触发Lambda函数,该函数将数据传递到AWS Elastic Search,然后传递到Kibana。 I am following this blog https://aws.amazon.com/blogs/big-data/building-a-near-real-time-discovery-platform-with-aws/
我正在关注这个博客https://aws.amazon.com/blogs/big-data/building-a-near-real-time-discovery-platform-with-aws/
The data is loading into the S3 bucket correctly but it never arrives in Kibana, I believe this is because the Lambda function is not being triggered by events in S3 as I would have hoped (there are no invocations or logs). 数据已正确加载到S3存储桶中,但从未到达Kibana,我相信这是因为Lambda函数并未像我希望的那样被S3中的事件触发(没有调用或日志)。 I think this is because I have not set permissions properly.
我认为这是因为我没有正确设置权限。 The Lambda function can be invoked manually by the test event.
可以通过测试事件手动调用Lambda函数。
On the Lambda function page I chose an existing role which I called lambda_s3_exec_role which has the AWSLambdaExecute policy attached to it but I feel I'm missing something else more specific to S3. 在Lambda函数页面上,我选择了一个名为lambda_s3_exec_role的现有角色,该角色具有附加的AWSLambdaExecute策略,但是我感觉缺少其他一些针对S3的角色。 I have been unable to follow this line in the blog in the create lambda function section because I do not recognise those options:
我无法在创建lambda函数部分的博客中关注此行,因为我无法识别这些选项:
"10. Choose lambda_s3_exec_role (if this value does not exist, choose Create new role S3 execution role)." “ 10.选择lambda_s3_exec_role(如果该值不存在,请选择“创建新角色S3执行角色”)。
Can anyone help me create the appropriate role/policy for the Lambda function, or spot what the problem may be? 谁能帮助我为Lambda函数创建适当的角色/策略,或者发现问题所在?
From view permissions on the Lambda function I currently have: 从Lambda函数的查看权限中,我目前拥有:
FUNCTION POLICY 功能政策
{
"Version": "2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "****",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "****",
"Condition": {
"ArnLike": {
"AWS:SourceArn": "arn:aws:s3:::****"
}
}
}
]
}
EXECUTION ROLE 执行角色
{
"roleName": "lambda_s3_exec_role",
"policies": [
{
"document": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:*"
],
"Resource": "arn:aws:logs:*:*:*"
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::*"
}
]
},
"name": "AWSLambdaExecute",
"id": "****",
"type": "managed",
"arn": "arn:aws:iam::aws:policy/AWSLambdaExecute"
}
]
}
The permissions you have listed look OK so I am going to try provide some steps that might help find the issue as it is difficult to understand specifically where your issue might be. 您列出的权限看起来不错,因此我将尝试提供一些可能有助于发现问题的步骤,因为很难明确地了解您的问题所在。
lambda.amazonaws.com
lambda.amazonaws.com
的受信任实体具有信任关系 twitter/raw-data/
. twitter/raw-data/
。 If firehose is writing to a path that isn't the event prefix then the event won't be invoked. aws lambda invoke \
--invocation-type RequestResponse \
--function-name helloworld \
--region region \
--log-type Tail \
--payload file://dummy_event.json \
--profile adminuser \
outputfile.txt
dummy_event.json dummy_event.json
{
"Records":[
{
"eventVersion":"2.0",
"eventSource":"aws:s3",
"awsRegion":"us-west-2",
"eventTime":"1970-01-01T00:00:00.000Z",
"eventName":"ObjectCreated:Put",
"userIdentity":{
"principalId":"AIDAJDPLRKLG7UEXAMPLE"
},
"requestParameters":{
"sourceIPAddress":"127.0.0.1"
},
"responseElements":{
"x-amz-request-id":"C3D13FE58DE4C810",
"x-amz-id-2":"FMyUVURIY8/IgAtTv8xRjskZQpcIZ9KG4V5Wp6S7S/JRWeUWerMUE5JgHvANOjpD"
},
"s3":{
"s3SchemaVersion":"1.0",
"configurationId":"testConfigRule",
"bucket":{
"name":"sourcebucket",
"ownerIdentity":{
"principalId":"A3NL1KOZZKExample"
},
"arn":"arn:aws:s3:::sourcebucket"
},
"object":{
"key":"HappyFace.jpg",
"size":1024,
"eTag":"d41d8cd98f00b204e9800998ecf8427e",
"versionId":"096fKKXTRTtl3on89fVO.nfljtsv6qko"
}
}
}
]
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.