AWS Java SDK 錯誤：請求簽名“與提供的簽名不匹配”檢查您的密鑰等。但憑證是正確的

Question

更新：這是我的愚蠢的復制/粘貼錯誤。 對此我將嘗試解釋的其他一些有趣的方面......

原始問題如下：

我試圖編寫一個將 JSON 寫入 S3 的 Spark 程序並遇到此錯誤：我們計算的請求簽名與您提供的簽名不匹配。 檢查您的密鑰和簽名方法。

所以我決定嘗試以最簡單的方式讓事情工作，使用 AWS Java SDK 和此處描述的示例程序

在 ~/.aws/credentials 中正確提供了我的憑據，上面引用的示例代碼無需修改即可工作。

然后我決定通過將 ~/.aws/credentials 移到一邊並重新運行來破壞事情，當然它出錯了。 然后我嘗試通過明確提供憑據來修復。 您可以在“FULL MODIFIED PROGRAM”部分看到稍微修改的示例程序的完整代碼。

但是我嘗試過的本質在下面立即給出。 請注意，我通過以下方式提供憑據

.withCredentials(provider)

在 s3Client 構建器上。 我還嘗試使用注釋掉並啟用下面的行來運行：

//.withClientConfiguration(clientConfiguration)

.

   public static void main(String[] args) throws IOException {
        String bucketName = "cbedford-sqlshackdemocli-test-only";
        String stringObjKeyName = "durango.saturday1";
        String fileObjKeyName = "clean.money.txt";
        String fileName = "/tmp/x";

        ClientConfiguration clientConfiguration = new ClientConfiguration();
        // Solution is update the Signer Version.
        clientConfiguration.setSignerOverride("S3SignerType");


        AWSCredentialsProvider provider = new  AWSCredentialsProvider() {
            @Override
            public AWSCredentials getCredentials() {
                return new AWSCredentials () {
                    @Override
                    public String getAWSAccessKeyId() {
                        // String below copied form ~/.aws/credentials 
                        // [default]
                        // aws_access_key_id = somethingsomething
                        return "somethingsomething";        
                    }
                    @Override
                    public String getAWSSecretKey() {
                        // String below copied form ~/.aws/credentials 
                        // [default]
                        // aws_secret_access_key = somethingsomethingelse
                        return "somethingsomethingelse";
                    }
                };
            }
            @Override
            public void refresh() {
            }
        };

        try {
            //This code expects that you have AWS credentials set up per:
            // https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html
            AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
                    .withRegion(Regions.US_EAST_1)
                    .withCredentials(provider)
                    //.withClientConfiguration(clientConfiguration)
                    .build();

無論我嘗試什么，我都未能通過內聯方法提供與 my.aws 目錄中相同的憑據。 如果有人能發現我的愚蠢錯誤（我知道我一定犯了一個。）我將非常感激。

完全修改程序

import com.amazonaws.AmazonServiceException;
import com.amazonaws.ClientConfiguration;
import com.amazonaws.SdkClientException;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.ObjectMetadata;
import com.amazonaws.services.s3.model.PutObjectRequest;

import java.io.File;
import java.io.IOException;

public class UploadObject {

    public static void main(String[] args) throws IOException {
        String bucketName = "cbedford-sqlshackdemocli-test-only";
        String stringObjKeyName = "durango.saturday1";
        String fileObjKeyName = "clean.money.txt";
        String fileName = "/tmp/x";

        ClientConfiguration clientConfiguration = new ClientConfiguration();
        // Solution is update the Signer Version.
        clientConfiguration.setSignerOverride("S3SignerType");


        AWSCredentialsProvider provider = new  AWSCredentialsProvider() {
            @Override
            public AWSCredentials getCredentials() {
                return new AWSCredentials () {
                    @Override
                    public String getAWSAccessKeyId() {
                        return "somethingsomething";        

                    }
                    @Override
                    public String getAWSSecretKey() {
                        return "somethingsomethingelse";        
                    }
                };
            }
            @Override
            public void refresh() {
            }
        };

        try {
            //This code expects that you have AWS credentials set up per:
            // https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html
            AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
                    .withRegion(Regions.US_EAST_1)
                    .withCredentials(provider)
                    //.withClientConfiguration(clientConfiguration)
                    .build();

            // Upload a text string as a new object.
            s3Client.putObject(bucketName, stringObjKeyName, "Uploaded String Object");

            // Upload a file as a new object with ContentType and title specified.
            PutObjectRequest request = new PutObjectRequest(bucketName, fileObjKeyName, new File(fileName));
            ObjectMetadata metadata = new ObjectMetadata();
            metadata.setContentType("plain/text");
            metadata.addUserMetadata("title", "someTitle");
            request.setMetadata(metadata);
            s3Client.putObject(request);
        } catch (AmazonServiceException e) {
            // The call was transmitted successfully, but Amazon S3 couldn't process
            // it, so it returned an error response.
            e.printStackTrace();
        } catch (SdkClientException e) {
            // Amazon S3 couldn't be contacted for a response, or the client
            // couldn't parse the response from Amazon S3.
            e.printStackTrace();
        }
    }
}

Answer 1

可以與 v4 簽名相關，您需要將請求路由到存儲桶所在的特定 AWS 區域。 最近的地區（倫敦、法蘭克福等）僅是 v4 簽名。

why not add the hadoop-aws jar for the exact hadoop versions you have in your lib dir *and the exact aws JARs it was built with, and try to use the s3a connector, to see if it works or, if it doesn't ，提供了哪些錯誤。 然后查看 hadoop 文檔上的 Troubleshooting_s3a 文檔，看看可能的原因是什么。