Wget Public S3存储桶

Question

I'd like to download all of the files in this public s3 bucket without an AWS access key (since this is part of a tutorial that others have to follow). Each of the constituent files ( example ) can be downloaded individually.

Answer 1

You're not taking an easy road:

1. you will need to get the key (ie files) in the bucket

you can use rest API and make a request as follow

    curl -H "GET /?list-type=2 HTTP/1.1" \
    -H "Host: halitereplaybucket.s3.amazonaws.com" \
    -H "Date: 20161025T124500Z" \
    -H "Content-Type: text/plain" https://halitereplaybucket.s3.amazonaws.com/

This will return a response in xml format with the contents and all keys from the bucket.

2. You will need to parse the response and extract all keys

3. for each of the keys, you can make a request to download the file

    wget 'http://halitereplaybucket.s3.amazonaws.com/{file}' 

I understand you mention you do not want to use a AWS key but using the CLI it will be as simple as

aws s3 sync s3://halitereplaybucket .