SignatureDoesNotMatch when uploading to AWS S3

Question

Since I've searched long to find the bug, I want to share it here, in case somebody runs into the same problem: I've implemented the following code snippet to upload data to S3:

public class S3FileUploadHandler {
  private static final String USER_META_DATA_KEY_MODIFIED = "last-modified";
  private static final SimpleDateFormat sdf = new SimpleDateFormat("EEE MMM dd HH:mm:ss zzz yyyy");
  private final AmazonS3Client s3;
  private final TransferManager tm;
  private final String bucketName = "myTestBucket";
  ...

    public void run() {
        File file = ... // retrieve file to upload
        FileInputStream fis = null;
        try {
            fis = new FileInputStream(file);
            final String key = ... // some path on S3
            final ObjectMetadata objectMetadata = buildObjectMetadata(file);
            final PutObjectRequest req = new PutObjectRequest(bucketName, key, fis, objectMetadata);
            final Upload upload = tm.upload(req);
            upload.addProgressListener(new MyProgressListener(upload));
            upload.waitForCompletion();
        } catch (final AmazonServiceExceptionase) {
            logger.debug("Couldn't put file {}. AmazonServiceException {}", file, ase);
            logger.error("Couldn't put file {}. AmazonServiceException Error Message: {}", file,    ase.getMessage());
            logger.error("HTTP Status Code: " + ase.getStatusCode());
            logger.error("AWS Error Code:   " + ase.getErrorCode());
            logger.error("Error Type:       " + ase.getErrorType());
            logger.error("Request ID:       " + ase.getRequestId());
        } catch (final Exception e) {
            ...
        } finally {
            if (fis != null) {
                fis.close();
            }
        }
    }

    private ObjectMetadata buildObjectMetadata(final File file) {
        final ObjectMetadata metadata = new ObjectMetadata();
        metadata.setContentLength(file.length());
        final Date lastModifiedDate = new Date(file.lastModified());
        final String dateStr = sdf.format(lastModifiedDate);
        metadata.addUserMetadata(USER_META_DATA_KEY_MODIFIED, dateStr);
        return metadata;
    }
}

The code snipped uploads a file to AWS S3 and to retain the last modified timestamp of the file, I set this information as user-meta-data.

Somtimes the above code works, sometimes the upload breaks with the following exception:

Couldn't put file C:\Temp\xyz.txt. AmazonServiceException Error Message: The request signature we calculated does not match the signature you provided. Check your key and signing method. (Service: Amazon S3; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID: B8BC3251B9C1A3F5)
HTTP Status Code: 403
AWS Error Code:   SignatureDoesNotMatch
Error Type:       Client
Request ID:       B8BC3251B9C1A3F5

Answer 1

The bug in the code above is, that the SimpleDateFormat potentially contains non-ASCII characters.

Eg the last modified date on a German locale may be

Di Mär 21 10:29:00 MEZ 2017

The AWS documentation states that

User-defined metadata is a set of key-value pairs. Amazon S3 stores user-defined metadata keys in lowercase. Each key-value pair must conform to US-ASCII when using REST and UTF-8 when using SOAP or browser-based uploads via POST.

AWS could really work on a better Exception text here, but with the above information the fix is simple:

private static final SimpleDateFormat sdf = new SimpleDateFormat("EEE MMM dd HH:mm:ss zzz yyyy", Locale.US);

...resulting in (always) accepted meta-data strings

Tue Mar 21 10:29:00 CET 2017