[英]How to get contents of a text file from AWS s3 using a lambda function?
I was wondering if I could set up a lambda function for AWS, triggered whenever a new text file is uploaded into an s3 bucket.我想知道我是否可以为 AWS 设置 lambda function,只要将新文本文件上传到 s3 存储桶中就会触发。 In the function, I would like to get the contents of the text file and process it somehow.
在 function 中,我想获取文本文件的内容并以某种方式对其进行处理。 I was wondering if this was possible...?
我想知道这是否可能......?
For example, if I upload foo.txt, with contents foobarbaz, I would like to somehow get foobarbaz in my lambda function so I can do stuff with it.例如,如果我上传 foo.txt,内容为 foobarbaz,我想以某种方式在我的 lambda function 中获取 foobarbaz,以便我可以用它做一些事情。 I know I can get metadata from getObject, or a similar method.
我知道我可以从 getObject 或类似方法中获取元数据。
Thanks!谢谢!
The S3 object key and bucket name are passed into your Lambda function via the event parameter. S3 对象键和存储桶名称通过事件参数传递到您的 Lambda 函数中。 You can then get the object from S3 and read its contents.
然后,您可以从 S3 获取对象并读取其内容。
Basic code to retrieve bucket and object key from the Lambda event
is as follows:从 Lambda
event
检索存储桶和对象键的基本代码如下:
exports.handler = function(event, context, callback) {
const bkt = event.Records[0].s3.bucket.name;
const key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
};
Once you have the bucket and key, you can call getObject to retrieve the object:获得存储桶和密钥后,您可以调用 getObject 来检索对象:
const AWS = require('aws-sdk');
const s3 = new AWS.S3();
exports.handler = function(event, context, callback) {
// Retrieve the bucket & key for the uploaded S3 object that
// caused this Lambda function to be triggered
const Bucket = event.Records[0].s3.bucket.name;
const Key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
// Retrieve the object
s3.getObject({ Bucket, Key }, function(err, data) {
if (err) {
console.log(err, err.stack);
callback(err);
} else {
console.log("Raw text:\n" + data.Body.toString('ascii'));
callback(null, null);
}
});
};
Here's an updated JavaScript example using ES6-style code and promises, minus error-handling:这是一个使用 ES6 样式代码和 promise 的更新 JavaScript 示例,减去错误处理:
const AWS = require('aws-sdk');
const s3 = new AWS.S3();
exports.handler = async (event, context) => {
const Bucket = event.Records[0].s3.bucket.name;
const Key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
const data = await s3.getObject({ Bucket, Key }).promise();
console.log("Raw text:\n" + data.Body.toString('ascii'));
};
A number of posters have asked for the equivalent in Java, so here's an example:许多海报要求 Java 中的等价物,所以这里有一个例子:
package example;
import java.net.URLDecoder;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.lambda.runtime.events.S3Event;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.event.S3EventNotification.S3EventNotificationRecord;
public class S3GetTextBody implements RequestHandler<S3Event, String> {
public String handleRequest(S3Event s3event, Context context) {
try {
S3EventNotificationRecord record = s3event.getRecords().get(0);
// Retrieve the bucket & key for the uploaded S3 object that
// caused this Lambda function to be triggered
String bkt = record.getS3().getBucket().getName();
String key = record.getS3().getObject().getKey().replace('+', ' ');
key = URLDecoder.decode(key, "UTF-8");
// Read the source file as text
AmazonS3 s3Client = new AmazonS3Client();
String body = s3Client.getObjectAsString(bkt, key);
System.out.println("Body: " + body);
return "ok";
} catch (Exception e) {
System.err.println("Exception: " + e);
return "error";
}
}
}
You can use data.Body.toString('ascii')
to get the contents of the text file, assuming that the text file was encoded used ascii format.您可以使用
data.Body.toString('ascii')
来获取文本文件的内容,假设文本文件是使用 ascii 格式编码的。 You can also pass other encoding types to the function.您还可以将其他编码类型传递给函数。 Check out Node-Buffer for further details.
查看节点缓冲区以获取更多详细信息。
I am using lambda function with a python 3.6 environment.我在 python 3.6 环境中使用 lambda 函数。 The code below will read the contents of a file main.txt inside bucket my_s3_bucket.
下面的代码将读取存储桶 my_s3_bucket 中文件 main.txt 的内容。 Make sure to replace name of bucket and file name according to your needs.
确保根据您的需要替换存储桶名称和文件名。
def lambda_handler(event, context):
# TODO implement
import boto3
s3 = boto3.client('s3')
data = s3.get_object(Bucket='my_s3_bucket', Key='main.txt')
contents = data['Body'].read()
print(contents)
The new AWS SDK v3 means that the files are read back as a readable stream.新的 AWS SDK v3 意味着文件被读回为可读的 stream。 You'll need to take that into consideration from now on as well.
从现在开始,您也需要考虑到这一点。
https://carova.io/snippets/read-data-from-aws-s3-with-nodejshttps://carova.io/snippets/read-data-from-aws-s3-with-nodejs
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.