Limit number of file shares in AWS storage gateway for multiple local servers

Question

There are 9 local on premises servers with unique output. Seems like the AWS storage gateway will expose one share per AWS S3 bucket.

It means that there would be total of 9 shares with 9 S3 buckets. Is there any way to do this using just one file share?

Answer 1

You can setup a single s3 bucket as your data hub on aws and setup a nfs file-share for this bucket through storage gateway. Locally setup sub-folders for each data source/destination under this common nfs share.

Have servers/processes write to their own folders. Now all these folders will be replicated to the single s3 bucket

have a lambda function on s3 bucket that will replicate (aws-cli s3 sync) each of the folders to their corresponding target s3 bucket.

this approach will handle traffic for all your servers with a single bucket share. Storage gateway only supports 10 file-shares/10buckets per gateway instance. Above approach lets you go past that limit. All the best.