使用 java sdk 将带有用户元数据的大文件上传到 Amazon S3

Question

我正在尝试使用 Java API 上传大文件，但我还需要添加用户元数据。 如果我只是使用

TransferManager tm = new TransferManager(new BasicAWSCredentials(accessKey, secretKey));

Upload upload = tm.upload(AmazonS3Manager.bucketName, imageKey, file);
upload.waitForCompletion();

然后一切正常，但如果我使用：

ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentLength(file.length());
metadata.addUserMetadata("My key", "My value");
FileInputStream input = new FileInputStream(file);
Upload upload = tm.upload(AmazonS3Manager.bucketName, imageKey, input, metadata);

然后它不起作用，我在控制台中得到以下 output ：

Jul 5, 2011 4:33:15 PM com.amazonaws.http.AmazonHttpClient executeHelper
INFO: Sending Request: POST https://mybucket.s3.amazonaws.com /test.jpg Parameters: (uploads: null, ) Headers: (Content-Type: application/x-www-form-urlencoded; charset=utf-8, x-amz-meta-length: 312612077, )
Jul 5, 2011 4:33:16 PM com.amazonaws.http.AmazonHttpClient handleResponse
INFO: Received successful response: 200, AWS Request ID: 2A5B3538795CE730
Jul 5, 2011 4:33:16 PM com.amazonaws.http.AmazonHttpClient executeHelper
INFO: Sending Request: PUT https://mybucket.s3.amazonaws.com /test.jpg Parameters: (uploadId: rwUlbXqtRyMUWiVYKzGqRQH90fVLi9_|Secret Key Removed|_w--, partNumber: 1, ) Headers: (Content-Length: 5242880, Content-Type: application/x-www-form-urlencoded; charset=utf-8, )
Jul 5, 2011 4:34:00 PM com.amazonaws.http.AmazonHttpClient handleResponse
INFO: Received successful response: 200, AWS Request ID: 5E5AF291FBDBDD36
Jul 5, 2011 4:34:00 PM com.amazonaws.http.AmazonHttpClient executeHelper
INFO: Sending Request: PUT https://mybucket.s3.amazonaws.com /test.jpg Parameters: (uploadId: rwUlbXqtRyMUWiVYKzGqRQH90fVLi9_|Secret Key Removed|_w--, partNumber: 2, ) Headers: (Content-Length: 5242880, Content-Type: application/x-www-form-urlencoded; charset=utf-8, )
Jul 5, 2011 4:34:00 PM org.apache.http.impl.client.DefaultRequestDirector tryExecute
INFO: I/O exception (java.io.IOException) caught when processing request: Read error
Jul 5, 2011 4:34:00 PM org.apache.http.impl.client.DefaultRequestDirector tryExecute
INFO: Retrying request
Jul 5, 2011 4:34:00 PM com.amazonaws.http.AmazonHttpClient executeHelper
WARNING: Unable to execute HTTP request: null
Jul 5, 2011 4:34:00 PM com.amazonaws.http.AmazonHttpClient executeHelper
INFO: Sending Request: DELETE https://mybucket.s3.amazonaws.com /test.jpg Parameters: (uploadId: rwUlbXqtRyMUWiVYKzGqRQH90fVLi9_|Secret Key Removed|_w--, ) Headers: (Content-Type: application/x-www-form-urlencoded; charset=utf-8, )
Jul 5, 2011 4:34:01 PM com.amazonaws.http.AmazonHttpClient handleResponse
INFO: Received successful response: 204, AWS Request ID: 0EFC3F8D0FA6097E

非常感谢任何帮助！

Answer 1

您可以使用以下代码进行文件上传：

    ObjectMetadata metadata = new ObjectMetadata();
        metadata.setContentLength(file.length());
        metadata.addUserMetadata("My key", "My value");
        FileInputStream input = new FileInputStream(file);
        PutObjectRequest putObjectRequest = new PutObjectRequest(AmazonS3Manager.bucketName, imageKey, input, metadata);
s3client.putObject(putObjectRequest.withCannedAcl(CannedAccessControlList.PublicRead));

谢谢

Answer 2

您可能还需要设置内容类型，并且可能还需要设置校验和，因为它无法从输入 stream 中确定。 这有一些类似情况的信息http://docs.amazonwebservices.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3.html#putObject(com.amazonaws.services.s3.modelP )。

Answer 3

如果您尝试上传超过 5 GB 的文件并更新元数据，那么亚马逊不允许这样做。 因为 amazon s3 不提供超过 5Gb 文件的复制操作。

Amazon S3 最近开始支持多部分操作的复制操作，因此我还不能说什么。

https://forums.aws.amazon.com/thread.jspa?messageID=256605

谢谢