如何从 curl 生成正确的 RestTemplate 用法？

Question

I have a working curl command that is below:

curl -v  -H "Content-Type:application/octet-stream" \
         -H "x-amz-server-side-encryption:aws:kms" \
         -H "x-amz-server-side-encryption-aws-kms-key-id:abcdef81-abcd-4c85-b1d8-ee540d0a5f5d" \
         --upload-file /Users/fd/Downloads/video.mp4 \
         'https://video-uploads-prod.s3-accelerate.amazonaws.com/ABCDEAQGZHEhM55fvvA/ads-aws_userUploadedVideo?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200106T165718Z&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-server-side-encryption%3Bx-amz-server-side-encryption-aws-kms-key-id&X-Amz-Expires=86400&X-Amz-Credential=ABCDEFHLWTCWZ2MUPPBQ%2F20200106%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=037949abcd1234b063c75d3d505dd9120dd3fa9250c1ababa152e91fee123ca0'

The curl is working properly:

* We are completely uploaded and fine
< HTTP/1.1 200 OK

However, when I try to use RestTemplate (i'm using spring boot 1.5.6) I'm not able to make it work. The code I use is:

byte[] media = //video in mp4//;
String uploadUrl = "https://video-uploads-prod.s3-accelerate.amazonaws.com/ABCDEAQGZHEhM55fvvA/ads-aws_userUploadedVideo?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200106T165718Z&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-server-side-encryption%3Bx-amz-server-side-encryption-aws-kms-key-id&X-Amz-Expires=86400&X-Amz-Credential=ABCDEFHLWTCWZ2MUPPBQ%2F20200106%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=037949abcd1234b063c75d3d505dd9120dd3fa9250c1ababa152e91fee123ca0";

RestTemplate restTemplate = new RestTemplate();

HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_OCTET_STREAM);
headers.set("x-amz-server-side-encryption", encryption);
headers.set("x-amz-server-side-encryption-aws-kms-key-id", awsKmsKeyId);

HttpEntity entity = new HttpEntity<>(media, headers);

ResponseEntity<String> respEntity = restTemplate.exchange(uploadUrl, HttpMethod.POST, entity, String.class);

The error I get from AWS is:

<Error>
   <Code>AuthorizationQueryParametersError</Code>
   <Message>Error parsing the X-Amz-Credential parameter; the Credential is mal-formed; expecting "&lt;YOUR-AKID&gt;/YYYYMMDD/REGION/SERVICE/aws4_request".</Message>
   <RequestId>51FF099744C43804</RequestId>
   <HostId>FOlLws+txYMP0hKEg7aDjQeeARdn7bJN+lw7q/aGA48hRnr1YEsJrVmRi6oEz+mkpHlTIax5MkI=</HostId>
</Error>

My suspicion is that RestTemplate is changing the encoding of the URL. Is there anyway to replicate exactly the same as curl with RestTemplate?

Answer 1

I haven't test the following approach but what if you pass an expanded url to a restTemplate instance?

String uploadUrl = "...{your_params_in_placeholders}";
URI expanded = new UriTemplate(url).expand(uploadUrl, <param_values>);
url = URLDecoder.decode(expanded.toString(), "UTF-8");
restTemplate.exchange(url, HttpMethod.POST, entity, String.class);

If it doesn't work you can try to pay some attention to the encoded values in your URL(I mean "%2F" and so on)

Answer 2

I could make it work after struggling a lot with this. The code is pretty ugly but made the trick to work. Post for others in case you need it. Anyway, this is not answering my question I want get the exact RestTemplate configuration out of a curl command .

I had to manually decode each query param with URLDecoder.decode and then built the URI with UriComponentsBuilder.fromHttpUrl and added the previously decoded params

byte[] media = //video in mp4//;
String uploadUrl = "https://video-uploads-prod.s3-accelerate.amazonaws.com/ABCDEAQGZHEhM55fvvA/ads-aws_userUploadedVideo?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200106T165718Z&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-server-side-encryption%3Bx-amz-server-side-encryption-aws-kms-key-id&X-Amz-Expires=86400&X-Amz-Credential=ABCDEFHLWTCWZ2MUPPBQ%2F20200106%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=037949abcd1234b063c75d3d505dd9120dd3fa9250c1ababa152e91fee123ca0";

RestTemplate restTemplate = new RestTemplate();

HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_OCTET_STREAM);
headers.set("x-amz-server-side-encryption", encryption);
headers.set("x-amz-server-side-encryption-aws-kms-key-id", awsKmsKeyId);

HttpEntity entity = new HttpEntity<>(media, headers);

String url = uploadUrl.split("\\?")[0];
String[] urlParams = uploadUrl.split("\\?")[1].split("&");

MultiValueMap<String, String> params = new LinkedMultiValueMap<String, String>();

for (String p : urlParams) {
    String key = p.split("=")[0];
    String value = URLDecoder.decode(p.split("=")[1], StandardCharsets.UTF_8.toString());
    params.put(key, Collections.singletonList(value));
}

String uri = UriComponentsBuilder.fromHttpUrl(url)
        .queryParams(params)
        .build()
        .toUriString();

ResponseEntity<String> respEntity = restTemplate.exchange(uri, HttpMethod.PUT, entity, String.class);

Answer 3

You can pass a URI as opposed to a String in your RestTemplate and that should get rid of your error. Had the same thing happening here.

URI uri = URI.create(urlAsString);
Map response = new RestTemplate().exchange(uri, GET, new HttpEntity(headers), Map.class).getBody();

It seems like slashes are being hex-escaped when using a String so %2F becomes %252F . Passing in a URI directly avoids that.