How to create nested folders in aws s3 bucket using nodejs with lambda function?

Question

In my project I am using Angular 6, nodejs with lambda function and api gateway and below is my folder structure.

--Bucket Name
   |--Folder (folder name dynamically change based on user login)
         |----sub Folder (sub folder name dynamically change)
            |--- bird.jpg (file name also dynamic)

I have created bucket and folder using the below code and i need to create sub folder in folder and store jpg or text file. below is my code:

var AWS = require('aws-sdk');

AWS.config.update({
region: 'us-east-2',
accessKeyId: 'accessid',
secretAccessKey: 'secret id'
});
var s3 = new AWS.S3();

var bucketName = 'bucket name'; 

exports.handler = function uploadToS3(event, context, callback) {
s3.createBucket({Bucket: bucketName}, function() {

var params = {Bucket: bucketName,Key: event['keyName']};
s3.putObject(params, function(err, data) {
  if (err)
    console.log(err);
  else
    console.log("Successfully uploaded data to " + bucketName);
  });
});
callback(null,{ result : 'SUCCESS'});
};

How to create sub folder in folder and store text or jpg value in subfolder??

Answer 1

In S3, there are no "sub folders", it is flat storage. It's just files in a bucket. However, if you include a forward slash in the file key, the AWS S3 Console (and other tools) will use that to display your files as if they were in "folders."

folder1/folder2/bird.jpg has two "prefixes", but is a single Key for a single file.

So just name it whatever you want, your "folder structure" is yours to create. And S3 stores it in the bucket!

Answer 2

If you directly upload to a path which does not exist, S3 will create the folders for you.

So if you upload for example, you have the following structure

folder --- > subfolder  -->  image

if you specify the path while uploading to be

/folder/subfolder/image.jpg

S3 will automatically create the folder and the subfolder.

Answer 3

There are no folders in S3. Every object within a bucket is referenced with its key. However, the AWS console (and other clients) choose to present the key as if each / in the key acts as a folder divider.

Copied from the S3.putObject() API documentation (scroll down):

var params = {
  Bucket: 'STRING_VALUE', /* required */
  Key: 'STRING_VALUE', /* required */
  Body: new Buffer('...') || 'STRING_VALUE' || streamObject,
  // other parameters omitted
};
s3.putObject(params, function(err, data) {
  if (err) console.log(err, err.stack); // an error occurred
  else     console.log(data);           // successful response
});

Scroll further down for the explanations:

Body — (Buffer, Typed Array, Blob, String, ReadableStream) Object data.

Bucket — (String) Name of the bucket to which the PUT operation was initiated.

Key — (String) Object key for which the PUT operation was initiated.

Translated params to your example:

var folder = // ...
var subFolder = // ...
var fileName = // ...
var fileContent = // ...

var params = {
  Bucket: bucketName,
  Key: `${folder}/${subFolder}/${fileName}`,
  Body: fileContent
};

---

Comments

• As a general rule, you should never store credentials (in this case the accessKeyId and secretAccessKey ) as part of your source code. In AWS Lambda, you associate the necessary permissions by creating a Lambda Execution Role that has been tailored to only create the specific actions required by the Lambda source code (in your example the s3:PutObject for S3 access and the managed policy arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole for CloudWatch logs.

• You are calling s3.createBucket() once per event, meaning that you (in contrast to your explanation) will attempt to create a new bucket for every Lambda invocation.

• You are calling the callback(...) before the s3.putObject() callback has been triggered. Consequently, your function will return before the it has completed. Move your callback call inside the s3.putObject() callback and make sure that you call the callback with an error parameter in case of failure to make sure that your function returns correctly.