Update amazon s3 object meta data with lambda without performing a object copy?

Question

Is it possible to add or update an s3 objects metadata with a lambda function without making a copy of the object? This 2 year old post says that we do need to make a copy , but its 2 years old, so maybe things have changed?

Answer 1

In one sense, nothing has changed, fundamentally, because objects are immutable, and metadata is part of the object. To "change" an immutable object requires a copy and replace.

However , S3 objects now support tagging, which is a different class of "metadata," attached to -- rather than part of -- the object.

S3 Object Tagging – You can associate multiple key-value pairs (tags) with each of your S3 objects, with ability to change them at any time. The tags can be used to manage and control access, set up S3 Lifecycle policies, customize the S3 Analytics, and filter the CloudWatch metrics. You can think of the bucket as a data lake, and use tags to create a taxonomy of the objects within the lake. This is more flexible than using the bucket and a prefix, and allows you to make semantic-style changes without renaming, moving, or copying objects.

— S3 Storage Managment Update, 2017-03-20

See also Difference Between Object Tags and Object Metadata?

Answer 2

use s3.headObject method and setup corresponding metadata via object.Metadata

I create lambda and subscribe it to event when new object is created:

console.log('Loading function');

const aws = require('aws-sdk');

const s3 = new aws.S3({ apiVersion: '2006-03-01' });


exports.handler = async (event, context, callback) => {
    console.log('Received event:', JSON.stringify(event, null, 2));

    const bucket = event.Records[0].s3.bucket.name;
    const key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
    var params = {
        Bucket: bucket,
        Key: key,
        CopySource: encodeURIComponent( bucket+"/"+key ),
        Metadata: {
            mode: '33188',
        },
        MetadataDirective: 'REPLACE',
    };
    try {
        var data =  await s3.copyObject(params).promise();
        console.log('UPDATE SUCCESS', data);
    } catch (err) {
        console.log(err);
        const message = `Error updating object ${key} from bucket ${bucket}. Make sure they exist and your bucket is in the same region as this function.`;
        console.log(message);
        throw new Error(message);
    }
};

WARNING Do not forget this:

Otherwise you will fall into infinite loop