List all objects in AWS S3 bucket

Question

I am trying to figure out how to list all the objects from an AWS S3 bucket in Swift. I can't seem to find the information anywhere on the internet, but maybe I didn't look hard enough. If anyone could refer me to the code that will allow me to do this that would be great.

Answer 1

Don't know if you still need it but here you go:

let credentialsProvider = AWSStaticCredentialsProvider(accessKey: "ACCESS KEY", secretKey: "SECRET KEY")
    let configuration = AWSServiceConfiguration(region: .USWest2, credentialsProvider: credentialsProvider)
    AWSS3.registerS3WithConfiguration(configuration, forKey: "defaultKey")
    let s3 = AWSS3.S3ForKey("defaultKey")

    let listRequest: AWSS3ListObjectsRequest = AWSS3ListObjectsRequest()
    listRequest.bucket = "BUCKET"

    s3.listObjects(listRequest).continueWithBlock { (task) -> AnyObject? in
        print("call returned")
        let listObjectsOutput = task.result;
        for object in (listObjectsOutput?.contents)! {

            print(object.key)
        }

        return nil
    }

(Thanks to Daniel for reminding me not to use deprecated code) ;)

Answer 2

Jan-Dawid's answer is great - however, if you want your Bucket NOT to be publicly available to everyone and still get the object-list then you have to do some more stuff (especially in the AWS settings) for this to happen (see list of things you have to do below....).

Or in other words, how do you get all the objects listed in Swift having your bucket set to publicly restricted as shown in this image (ie NO public ACCESS)

There is a way you can do that in Swift without AWS user-login setup needed in your app:

Create an iOS App or open the one you want your bucket-object list to appear:

Type the following code:

let myIdentityPoolId = "eu-central-1:368293ad-f278-35ae-2678-6d40d48de2g5"
let credentialsProvider:AWSCognitoCredentialsProvider = AWSCognitoCredentialsProvider(regionType: .YOUR_REGION, identityPoolId: myIdentityPoolId)
let configuration = AWSServiceConfiguration(region: .YOUR_REGION, credentialsProvider:credentialsProvider)
AWSServiceManager.default().defaultServiceConfiguration = configuration

AWSS3.register(with: configuration!, forKey: "defaultKey")
let s3 = AWSS3.s3(forKey: "defaultKey")

let listRequest: AWSS3ListObjectsRequest = AWSS3ListObjectsRequest()
listRequest.bucket = "YOUR_BUCKET_NAME"

s3.listObjects(listRequest).continueWith { (task) -> AnyObject? in
    for object in (task.result?.contents)! {
        print("Object key = \(object.key!)")
    }
    return nil
}

(of course, you need to do some cosmetics to handle the case where no results should appear - but I leave that up to you.

Do make sure you have AWSS3 imported:

import AWSS3

After that, you need to do a whole set of steps inside the AWS Service pages in order for your code to really retrieve the desired bucket-objects list. Here is a step-by-step guidance :

Roughly, do the following:

1. Create a S3 Bucket
2. Add items to it (the objects you want to have listed)
3. In the bucket Access-control turn off all read&write access (as in image above)
4. Go to AWS Cognito main page and choose: Manage Federated Identities

(By the way, all these AWS main-pages can best be found going to the AWS-Services page and typing the words "Cognito" or "IAM" or "S3" into the search bar. These 3 are all you need)

But let's continue....

5. Inside "Manage Federated Identities", create a new identity pool (ie provide a name - AND DON'T FORGET TO TAG "Enable access to unauthenticated identities" (you can view the details and leave them unchanged) -> Press Allow!

6. Little EXTRA: In case you mess anything up and want to restart: There is a slight chance that the above step-Nr.6 does not work a second time with the same name. In that case, I recommend going to the "IAM" main page and there, also delete the 2 roles connected to your identity pool (ie under Roles in IAM-main-page). After the deletion of the roles, you can restart creating an identity-pool!)

7. A second little EXTRA: Througout all we do right here, please make sure that all your AWS-main-service pages are within the same region !!! (normaly the AWS-webpages start with something like https://eu-central-1 . or https://us-west2 . etc. Make sure they are consistent throughout !

But let's continue:

8. After identity-pool creation you should get an id for it (ie something like: eu-central-1:368293ad-f278-35ae-2678-6d40d48de2g5 (make sure you keep note of this id)

9. Inside the "IAM"-main page, under the Roles menu, you find 2 roles (that were created by step-5-8) - click on the Unauth one of the two (you see there is an Auth and and Unauth one)

10. Once the Unauth is clicked you get to the Summary page: There please keep note of the Role ARN (something like arn:aws:iam::274937169435:role/Cognito_YOURIDPOOLNAMEUnauth_Role )

11. Having the id's in hand you kept, go to the AWS-Policy-Generator page.

12. There, start filling you the Policy-Generator Form:

    12a. Select S3 Bucket Policy as the type

    12b. Chose Effect Allow

    12c. In the prinicpal-field, paste the Role-ARN you obtained in Step-10 (something like arn:aws:iam::274937169435:role/Cognito_YOURIDPOOLNAMEUnauth_Role )

    12d. Amazon Service field should be Amazon S3 (leave tag deselected)

    12e. Actions should be ListObjects (or ListBuckets depending of what you want to do...)

    12f. The Amazon Resource Name (ARN) should be arn:aws:s3:::YOUR_BUCKET_NAME (if you want all files from your bucket eventually listed in Swift, then leave the blank>

    12g. Press Add statement button

    12h. Press Create Policy button (that should generate a JSON-file)

Something like that:

{
  "Id": "Policy2846208763429",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt98475983711245435",
      "Action": [
        "s3:ListBucket"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME",
      "Principal": {
        "AWS": [
          "arn:aws:iam::274937169435:role/Cognito_YOURIDPOOLNAMEUnauth_Role"
        ]
      }
    }
  ]
}

13. With this Policy JSON-file in hand, go back to your "S3" bucket main page

14. Inside "S3" main page, go under --> "Persmissions" and press the tab "Bucket Policy"

15. Replace anything written inside the Bucket Policy tab's editor window - and paste your Policy-JSON File (from Step 12h. into it)

16. After pasing the JSON make sure you press "Save"

---> HERE WE GO !! THE SWFIT CODE SHOULD NOW RETRIEVE ALL BUCKET FILES IN THE RETURN TASK ANSWER !!!!!

17. Optional: If you need any other AWS Services inside your iOS App, it is recommended that you use the following entry inside your info.plist file (providing the correct identity-pool id and region)

Step Nr17 is not really necessary if you only want to fetch the object list of your bucket.

    <key>AWS</key>
    <dict>
        <key>CredentialsProvider</key>
        <dict>
            <key>CognitoIdentity</key>
            <dict>
                <key>Default</key>
                <dict>
                    <key>PoolId</key>
                    <string>eu-central-1:368293ad-f278-35ae-2678-6d40d48de2g5</string>
                    <key>Region</key>
                    <string>eu-central-1</string>
                </dict>
            </dict>
        </dict>
        <key>S3TransferUtility</key>
        <dict>
            <key>Default</key>
            <dict>
                <key>Region</key>
                <string>eu-central-1</string>
            </dict>
        </dict>
    </dict>