将 Java 对象存储到 Amazon S3 存储桶

Question

我有一个 Java 对象。 我得到这个作为我们系统底层 API 的响应。 我需要将此对象存储在 S3 存储桶内的文件夹中。

我发现了这个https://stackoverflow.com/a/41017427/9715520 。 但是，正如我所说，我需要将对象上传到存储桶内的文件夹中。 有没有办法我也可以指定文件夹名称。 该示例讲述了如何将对象保存到 S3 存储桶。 如果一个存储桶中有多个文件夹并且需要存储在特定文件夹中怎么办。 我们有什么办法吗？

我就是这样做的。 下面的代码是否足够，或者我是否需要明确创建 s3 客户端来上传对象？

private void s3Upload(MyObject myObject) {
         
         TransferManager transferManager = TransferManagerBuilder.standard().build();
         
         ObjectMapper objectMapper = new ObjectMapper(); 
         byte[] bytesToWrite = objectMapper.writeValueAsBytes(myObject);

         ObjectMetadata omd = new ObjectMetadata();
         omd.setContentLength(bytesToWrite.length);
         String uniqueID = UUID.randomUUID().toString();
         transferManager.upload(<bucketName>, uniqueID, new ByteArrayInputStream(bytesToWrite), omd);
    }

Answer 1

要上传对象（例如 PDF），建议您使用AWS SDK for Java V2 。 使用此 API，您有 2 个选择。

1. 您可以使用S3Client对象将对象上传到 Amazon S3 存储桶。

2. 您可以使用TransferManager对象将对象上传到 Amazon S3 存储桶。

Amazon S3 控制台支持将文件夹概念作为对对象进行分组的一种方式。 例如，如果要将 PDF 文件存储在名为folder2的文件夹中，请在objectKey中指定名称。 例如：

字符串 objectKey = "folder2/myPDF.pdf"

当您运行此代码时，myPDF 位于文件夹 2 中——如此屏幕截图所示：

如果您不熟悉使用适用于 Java V2 的AWS 开发工具包，请参阅Java V2 开发指南了解您需要的所有概念。 例如，如何使用凭证。

S3客户端

以下代码向您展示了如何使用此对象上传内容。

package com.example.s3;

// snippet-start:[s3.java2.s3_object_upload.import]
import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.model.PutObjectResponse;
import software.amazon.awssdk.services.s3.model.S3Exception;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
// snippet-end:[s3.java2.s3_object_upload.import]

/**
 * Before running this Java V2 code example, set up your development environment, including your credentials.
 *
 * For more information, see the following documentation topic:
 *
 * https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
 */

public class PutObject {

    public static void main(String[] args) {
        final String usage = "\n" +
                "Usage:\n" +
                "  <bucketName> <objectKey> <objectPath> \n\n" +
                "Where:\n" +
                "  bucketName - The Amazon S3 bucket to upload an object into.\n" +
                "  objectKey - The object to upload (for example, book.pdf).\n" +
                "  objectPath - The path where the file is located (for example, C:/AWS/book2.pdf). \n\n" ;

        if (args.length != 3) {
            System.out.println(usage);
            System.exit(1);
        }

        String bucketName =args[0];
        String objectKey = args[1];
        String objectPath = args[2];
        System.out.println("Putting object " + objectKey +" into bucket "+bucketName);
        System.out.println("  in bucket: " + bucketName);

        ProfileCredentialsProvider credentialsProvider = ProfileCredentialsProvider.create();
        Region region = Region.US_EAST_1;
        S3Client s3 = S3Client.builder()
                .region(region)
                .credentialsProvider(credentialsProvider)
                .build();

        String result = putS3Object(s3, bucketName, objectKey, objectPath);
        System.out.println("Tag information: "+result);
        s3.close();
    }

    // snippet-start:[s3.java2.s3_object_upload.main]
    public static String putS3Object(S3Client s3,
                                     String bucketName,
                                     String objectKey,
                                     String objectPath) {

        try {

            Map<String, String> metadata = new HashMap<>();
            metadata.put("x-amz-meta-myVal", "test");

            PutObjectRequest putOb = PutObjectRequest.builder()
                    .bucket(bucketName)
                    .key(objectKey)
                    .metadata(metadata)
                    .build();

            PutObjectResponse response = s3.putObject(putOb,
                    RequestBody.fromBytes(getObjectFile(objectPath)));

           return response.eTag();

        } catch (S3Exception e) {
            System.err.println(e.getMessage());
            System.exit(1);
        }
        return "";
    }

    // Return a byte array.
    private static byte[] getObjectFile(String filePath) {

        FileInputStream fileInputStream = null;
        byte[] bytesArray = null;

        try {
            File file = new File(filePath);
            bytesArray = new byte[(int) file.length()];
            fileInputStream = new FileInputStream(file);
            fileInputStream.read(bytesArray);

        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            if (fileInputStream != null) {
                try {
                    fileInputStream.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
        return bytesArray;
    }
    // snippet-end:[s3.java2.s3_object_upload.main]
}

TransferManager 对象

以下代码向您展示了如何使用此 Java 对象将内容上传到 S3 存储桶。

package com.example.transfermanager;

import software.amazon.awssdk.auth.credentials.EnvironmentVariableCredentialsProvider;
import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.transfer.s3.FileUpload;
import software.amazon.awssdk.transfer.s3.S3TransferManager;
import java.nio.file.Paths;

/**
 * Before running this Java V2 code example, set up your development environment, including your credentials.
 *
 * For more information, see the following documentation topic:
 *
 * https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
 */
public class UploadObject {

    public static void main(String[] args) {

        final String usage = "\n" +
                "Usage:\n" +
                "  <bucketName> <objectKey> <objectPath> \n\n" +
                "Where:\n" +
                "  bucketName - The Amazon S3 bucket to upload an object into.\n" +
                "  objectKey - The object to upload (for example, book.pdf).\n" +
                "  objectPath - The path where the file is located (for example, C:/AWS/book2.pdf). \n\n" ;

       if (args.length != 3) {
            System.out.println(usage);
            System.exit(1);
       }

        long mb = 1024;
        String bucketName = args[0];
        String objectKey = args[1];
        String objectPath = args[2];


        System.out.println("Putting an object into bucket "+bucketName +" using the S3TransferManager");
        ProfileCredentialsProvider credentialsProvider = ProfileCredentialsProvider.create();
        Region region = Region.US_EAST_1;
        S3TransferManager transferManager = S3TransferManager.builder()
                .s3ClientConfiguration(cfg ->cfg.region(region)
                        .credentialsProvider(credentialsProvider)
                        .targetThroughputInGbps(20.0)
                        .minimumPartSizeInBytes(10 * mb))
                .build();

        uploadObjectTM(transferManager, bucketName, objectKey, objectPath);
        System.out.println("Object was successfully uploaded using the Transfer Manager.");
        transferManager.close();
    }

    public static void uploadObjectTM( S3TransferManager transferManager, String bucketName, String objectKey, String objectPath) {

        FileUpload upload =
                transferManager.uploadFile(u -> u.source(Paths.get(objectPath))
                        .putObjectRequest(p -> p.bucket(bucketName).key(objectKey)));
        upload.completionFuture().join();
    }
}

另一个更新

看这行代码：

PutObjectResponse 响应 = s3.putObject(putOb, RequestBody.fromBytes (getObjectFile(objectPath)));

只要您获得要上传到字节数组的数据，就可以使用此代码将其上传到 Amazon S3。 我以物理 PDF 文件为例。 但是，如果您从另一个来源获得 byte[]，它仍然可以工作。

RequestBody 也支持这些 Javadocs 中讨论的其他方法。

https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/sync/RequestBody.html

希望这可以消除您的疑问。