Download Inventory File from AWS S3

Question

I have configured AWS to automatically create an inventory file, in the following location. Just to explain an inventory file is a csv.gz file with a directory listing of an entire bucket (named Bucket1). On a weekly basis a file with a gibberish name is created here:

Location: Bucket2\Inventory\Data\filename_gibberish.csv.gz

Question: I wish for my bash script to locate the latest inventory file based on the "last modified" time-stamp on AWS. Then check if my local directory has this file? If not, download it.

Pls help!

Answer 1

This command will list the most recently created object:

aws s3api list-objects --bucket my-bucket --prefix Inventory/Data/ --query 'sort_by(Contents, &LastModified)[-1].Key' --output text

It can be combined with a copy command:

aws s3 cp s3://my-bucket/$(aws s3api list-objects --bucket my-bucket --prefix Inventory/Data/ --query 'sort_by(Contents, &LastModified)[-1].Key' --output text) inventory.csv.gz

(Note that the bucket name appears twice in that command.)