简体   繁体   English

当新对象到达S3存储桶时触发Lambda函数

[英]Trigger Lambda function when a new object arrives in S3 bucket

I have S3 bucket named 'files'. 我有一个名为“文件”的S3存储桶。 Every day new file arrives there. 每天都有新文件到达。 Example: 例:

/files/data-01-23-2017--11-33am.txt
/files/data-01-24-2017--10-28am.txt

How would I make a Lambda function and set a trigger to execute one shell script on EC2 when new file arrives? 当新文件到达时,我将如何执行Lambda函数并设置触发器以在EC2上执行一个Shell脚本? Example of new file is: 新文件的示例是:

/files/data-01-25-2017--11-43am.txt

Command that I would want to execute on EC2 is (with parameter as new file name that just arrived): 我想在EC2上执行的命令是(使用参数作为刚到达的新文件名):

python /home/ec2-user/jobs/run_job.py data-01-25-2017--11-43am.txt

Amazon S3 can be configured to trigger an AWS Lambda function when a new object is created. 可以将Amazon S3配置为在创建新对象时触发AWS Lambda函数。 However, Lambda functions do not have access to your Amazon EC2 instances. 但是,Lambda函数无法访问您的Amazon EC2实例。 It is not an appropriate architecture to use. 它不是适合使用的体系结构。

Some alternative options (these are separate options, not multiple steps): 一些替代选项 (这些是单独的选项,而不是多个步骤):

  • Instead of running a command on an Amazon EC2 instance, put your code in the Lambda function (no EC2 instance required). 无需在Amazon EC2实例上运行命令,而是将代码放入Lambda函数 (无需EC2实例)。 (Best option!) (最佳选择!)
  • Configure Amazon S3 to push a message into an Amazon SQS queue . 配置Amazon S3以将消息推送到Amazon SQS队列中 Have your code on the EC2 instance regularly poll the queue. 让您的代码在EC2实例上定期轮询队列。 When it receives a message, process the object in S3. 收到消息后,在S3中处理对象。
  • Configure Amazon S3 to send a message to an Amazon SNS topic . 配置Amazon S3以将消息发送到Amazon SNS主题 Subscribe an end-point of your application (effectively an API) to the SNS queue, so that it receives a message when a new object has been created. 将您的应用程序的端点(实际上是API)订阅到SNS队列,以便在创建新对象时收到消息。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 当新文件到达S3时,触发luigi任务 - When a new file arrives in S3, trigger luigi task 从 S3 中删除 object 时触发 Lambda function - Trigger Lambda function when object is deleted from S3 AWS Lambda 在 S3 存储桶上触发 Neptune“无法为源启动新负载” - AWS Lambda trigger on S3 bucket to Neptune "Failed to start new load for the source" 根据将文件从一个 S3 存储桶复制到另一个存储桶的清单文件触发 AWS Lambda function - Trigger AWS Lambda function based on a manifest file which copies files from one S3 bucket to another python cdk 在 s3 存储桶上创建 lambda 触发器 - python cdk to create lambda trigger on s3 bucket 使用Python处理Lambda中的S3 Bucket Trigger事件 - Handling S3 Bucket Trigger Event in Lambda Using Python AWS Lambda和S3和Pandas-将CSV加载到S3中,触发Lambda,加载到pandas,放回存储桶中? - AWS Lambda and S3 and Pandas - Load CSV into S3, trigger Lambda, load into pandas, put back in bucket? Python 中的 AWS Lambda 将新文件复制到另一个 s3 存储桶 - AWS Lambda in Python to copy new files to another s3 bucket Lambda function 使用 Boto 删除 S3 存储桶 - Lambda function to delete an S3 bucket using Boto 编写 lambda function 将文件上传到我的 S3 存储桶 - Writing a lambda function to upload files to my S3 bucket
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM