在Java中从Amazon S3读取文件的最佳方式

Question

I am being given a URL to an S3 file in the following format:

s3n://XXXXXXXXXXXXXXXXXXXXXXX:XXXXXXXXXXXXXXXXXXXXXXXXXXXXX@bucket/foo/bar.json

What is the best way to go about pulling the json file from this location?

I would like to treat the URL as an opaque value, and avoid parsing it so that it can change with out me needing to change the underlying code.

When I attempt to pass the value to java.net.URL it throws a Mal Formed Exception, because it doesn't recognize the S3 protocol.

Thank you!

Answer 1

You can implement an URLStreamHandler that support the s3n protocol.

The Java doc of the URL constructor explain how URLStreamHandle are discovered.

In the URLStreamHandler you will need to do the url parsing to map it to need of the s3 client.

But any code using it doesn't have to be dependent on any S3 related element and should work with other storage if needed URLStreamHandler is present.

Answer 2

In Amazon S3 you first need to authenticate to the given bucket and then pull the InputStream of the needed object and then you can do your need.

To pull the object via Amazon RestWebService to get the object or to access the object via the URL you need to provide a public access to it. See here