AWS S3 cli download the latest 2 or more files onto directory

Question

I want to download the latest 2 files from an existing bucket folder, can anyone advise me how do I go about implementing this in a shellscript?

I know that the below command list the top 2 latest file in a directory, but how do I go about using aws cp to download the latest 2 files to my directory?

aws s3 ls $BUCKET --recursive | sort | tail -n 2 | awk '{print $4}' 

Answer 1

Something like this ought to do it:

LIST=$(aws s3 ls $BUCKET --recursive | sort | tail -n 2 | awk '{print $4}')

for x in $LIST; do
    aws s3 cp s3://$BUCKET/$x .
done

Answer 2

Using tail here is unnecessary if you use a reverse sort and force awk to only process the first 2 records. You can also use awks built in system function to save looping.

aws s3 ls $BUCKET --recursive | sort -r | awk 'NR<=2 { system("aws s3 ls "$4"/$x .")

Note the importance of printing the command before using system to ensure that the command is constructed correctly before execution.