如何从Amazon S3下载文件？

Question

I have a folder named output inside a bucket named BucketA . I have a list of files in output folder. How do I download them to my local machine using AWS Java SDK ?

Below is my code:

AmazonS3Client s3Client = new AmazonS3Client(credentials);
        File localFile = new File("/home/abc/Desktop/AmazonS3/");
        s3Client.getObject(new GetObjectRequest("bucketA", "/bucketA/output/"), localFile);

And I got the error:

AmazonS3Exception: The specified key does not exist.

Answer 1

Keep in mind that S3 is not a filesystem, but it is an object store . There's a huge difference between the two, one being that directory-style activities simply won't work.

Suppose you have an S3 bucket with two objects in it:

/path/to/file1.txt
/path/to/file2.txt

When working with these objects you can't simply refer to /path/to/ like you can when working with files in a filesystem directory. That's because /path/to/ is not a directory but just part of a key in a very large hash table. This is why the error message indicates an issue with a key . These are not filename paths but keys to objects within the object store.

In order to copy all the files in a location like /path/to/ you need to perform it in multiple steps. First, you need to get a listing of all the objects whose keys begin with /path/to , then you need to loop through each individual object and copy them one by one.

Here is a similar question with an answer that shows how to download multiple files from S3 using Java.

Answer 2

I know this question was asked longtime ago, but still this answer might help some one.

You might want to use something like this to download objects from S3

  new ListObjectsV2Request().withBucketName("bucketName").withDelimiter("delimiter").withPrefix("path/to/image/"); 

as mentioned in the S3 doc

delimiter be "/" and prefix be your "folder like structure" .

Answer 3

You can use the predefined classes for upload directory and download directory

For Download

MultipleFileDownload xfer = xfer_mgr.downloadDirectory(
    bucketName, key, new File("C:\\Users\\miracle\\Deskto\\Downloads"));

For Upload

MultipleFileUpload xfer = xfer_mgr.uploadDirectory(bucketName, key,Dir,true);

Answer 4

The error message means that the bucket (in this case "bucketA") does not contain a file with the name you specified (in this case "/bucketA/output/").

When you specify the key, do not include the bucket name in the key. S3 supports "folders" in the key, which are delimited with "/", so you probably do not want to try to use keys that end with "/".

If your bucket "bucketA" contains a file called "output", you probably want to say

new GetObjectRequest("bucketA", "output")

If this doesn't work, other things to check:

• Do the credentials you are using have permission to read from the bucket?
• Did you spell all the names correctly?

You might want to use listObjects("bucketA") to verify what the bucket actually contains (as seen with the credentials you are using).