[英]RestTemplate not working with S3 pre-signed put URLs
I have a server generate AWS S3 pre-signed PUT URLs and then I'm trying to uploading a byte[]
into that URL using RestTemplate with this code: 我有一个服务器生成AWS S3预签名PUT URL,然后尝试使用带有以下代码的RestTemplate将
byte[]
上传到该URL:
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.ALL));
HttpEntity<byte[]> entity = new HttpEntity<>("Testing testing testing".getBytes(), headers);
System.out.println(restTemplate.exchange(putUrl, HttpMethod.PUT, entity, String.class));
When I run that code, I get this error: 当我运行该代码时,出现以下错误:
Exception in thread "JavaFX Application Thread" org.springframework.web.client.HttpClientErrorException: 400 Bad Request
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:63)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:700)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:653)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:613)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:531)
at tech.dashman.dashman.controllers.RendererAppController.lambda$null$2(RendererAppController.java:95)
Unfortunately, there's nothing in the AWS S3 logs, so, I'm not sure what's going on. 不幸的是,AWS S3日志中没有任何内容,因此,我不确定发生了什么。 If I take that exact same URL and put it in the REST Client of IntelliJ IDEA, it just works (it creates an empty file in S3).
如果我使用完全相同的URL并将其放入IntelliJ IDEA的REST客户端中,它将正常工作(它将在S3中创建一个空文件)。
Any ideas what's wrong with my Java code? 任何想法我的Java代码有什么问题吗?
Here's a full example that does the signing and tries to uploading a small payload to S3: 这是一个完整的示例,它进行签名并尝试将小的有效负载上传到S3:
import com.amazonaws.HttpMethod;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.GeneratePresignedUrlRequest;
import org.joda.time.DateTime;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.web.client.RestTemplate;
import java.util.Date;
public class S3PutIssue {
static public void main(String[] args) {
String awsAccessKeyId = "";
String awsSecretKey = "";
String awsRegion = "";
String path = "";
String awsBucketName = "";
BasicAWSCredentials awsCredentials = new BasicAWSCredentials(awsAccessKeyId, awsSecretKey);
AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withRegion(awsRegion).
withCredentials(new AWSStaticCredentialsProvider(awsCredentials)).build();
Date expiration = new DateTime().plusDays(1).toDate();
GeneratePresignedUrlRequest urlRequest = new GeneratePresignedUrlRequest(awsBucketName, path);
urlRequest.setMethod(HttpMethod.PUT);
urlRequest.setExpiration(expiration);
String putUrl = s3Client.generatePresignedUrl(urlRequest).toString();
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
HttpEntity<byte[]> entity = new HttpEntity<>("Testing testing testing".getBytes(), headers);
restTemplate.exchange(putUrl, org.springframework.http.HttpMethod.PUT, entity, Void.class);
}
}
Do not convert your URL to String. 不要将您的URL转换为String。 Instead convert it to URI.
而是将其转换为URI。 I think there are some encoding issues when you converted to String.
我认为转换为String时存在一些编码问题。 For example the URL in String format had
%252F
, where it should have just been %2F
. 例如,字符串格式的URL具有
%252F
,而该位置应仅为%2F
。 Looks like some sort of double encoding issue. 看起来有点双重编码问题。
Leave as URL... 保留为URL ...
URL putUrl = amazonS3Client.generatePresignedUrl(urlRequest);
Convert to URI... 转换为URI ...
ResponseEntity<String> re = restTemplate.exchange(putUrl.toURI(), org.springframework.http.HttpMethod.PUT, entity, String.class);
EDIT: More info to clarify what is going on. 编辑:更多信息以澄清发生了什么。
The problem that occurred here is that when you call URL.toString()
in this instance you are given back an encoded String representation of the URL. 此处发生的问题是,在这种情况下,当您调用
URL.toString()
,您将获得URL的编码字符串表示形式。 But the RestTemplate is expecting a String url that is not yet encoded. 但是RestTemplate期望一个尚未编码的字符串url。 RestTemplate will do the encoding for you.
RestTemplate将为您进行编码。
For example look at the code below... 例如看下面的代码...
public static void main(String[] args) {
RestTemplate rt = new RestTemplate();
rt.exchange("http://foo.com/?var=<val>", HttpMethod.GET, HttpEntity.EMPTY, String.class);
}
When you run this you get the following debug message from Spring, notice how the url in the debug msg is encoded. 运行此命令时,您将从Spring获得以下调试消息,请注意调试消息中的url是如何编码的。
[main] DEBUG org.springframework.web.client.RestTemplate - Created GET request for "http://foo.com/?var=%3Cval%3E"
So you can see the RestTemplate will encode for you any String url's passed to it. 因此,您可以看到RestTemplate将为您编码传递给它的任何字符串url。 But the URL provided by AmazonS3Client is already encoded.
但是AmazonS3Client提供的URL已被编码。 See the code below.
请参见下面的代码。
URL putUrl = amazonS3Client.generatePresignedUrl(urlRequest);
System.out.println("putUrl.toString = " + putUrl.toString());
This prints out a String that is already encoded. 这将打印出一个已经编码的字符串。
https://private.s3.amazonaws.com/testing/?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20171114T191829Z&X-Amz-SignedHeaders=host&X-Amz-Expires=0&X-Amz-Credential=AKIAIJ7ZSL22IJTM6NTQ%2F20171114%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=eca611ea33d9ad5710207568dcf181e4318ce39271fd0f1ce05bd99ebbf4097
So when I stick that into the exchange method of the RestTemplate I get the following debug message. 因此,当我将其放入RestTemplate的交换方法中时,会收到以下调试消息。
[main] DEBUG org.springframework.web.client.RestTemplate - PUT request for "https://turretmaster.s3.amazonaws.com/testing/?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20171114T191829Z&X-Amz-SignedHeaders=host&X-Amz-Expires=0&X-Amz-Credential=AKIAIJ7ZSL22IJTM6NTQ%252F20171114%252Fus-east-1%252Fs3%252Faws4_request&X-Amz-Signature=eca611ea33d9ad5710207568dcf181e4318ce39271fd0f1ce05bd99ebbf40975"
Notice how every %2F
from the url String turned into %252F
. 请注意,URL字符串中的每个
%2F
变成了%252F
。 %2F
is the encoded representation of /
. %2F
是/
的编码表示形式。 But %25
is %
. 但是
%25
是%
。 So it encoded a url that was already encoded. 因此,它对已经编码的网址进行了编码。 The solution was to pass a
URI
object to RestTemplate.exchange
, instead of an encoded String url. 解决方案是将
URI
对象传递给RestTemplate.exchange
,而不是编码的String url。
The source of issue is a double encoding of url characters. 问题的根源是网址字符的双重编码。 There are
/
in extended secret key which are encoded as %2
by s3Client.generatePresignedUrl
. 扩展秘密密钥中有
/
,由s3Client.generatePresignedUrl
编码为%2
。 When already encoded string is passed to restTemplate.exchange
it's internally converted to URI and encoded for the second time as %252
by UriTemplateHandler
in RestTemplate
source code. 当已经编码的字符串传递到
restTemplate.exchange
它将在内部转换为URI,并由RestTemplate
源代码中的UriTemplateHandler
第二次编码为%252
。
@Override
@Nullable
public <T> T execute(String url, HttpMethod method, @Nullable RequestCallback requestCallback,
@Nullable ResponseExtractor<T> responseExtractor, Object... uriVariables) throws RestClientException {
URI expanded = getUriTemplateHandler().expand(url, uriVariables);
return doExecute(expanded, method, requestCallback, responseExtractor);
}
So the easiest solution is to convert URL to URI using URL.toURI()
. 因此,最简单的解决方案是使用
URL.toURI()
将URL转换为URI。 If you don't have URI
and have String
when RestTemplate
is invoked then two options are possible. 如果在调用
RestTemplate
时没有URI
且没有String
,则可以有两个选择。
Pass URI instead for string to exchange method. 传递URI代替字符串来交换方法。
restTemplate.exchange(new URI(putUrl.toString()), HttpMethod.PUT, entity, Void.class);
Create default UriTemplateHandler
with NONE
encoding mode and pass it to RestTemplate
. 使用
NONE
编码模式创建默认的UriTemplateHandler
,并将其传递给RestTemplate
。
DefaultUriBuilderFactory defaultUriBuilderFactory = new DefaultUriBuilderFactory();
defaultUriBuilderFactory.setEncodingMode(DefaultUriBuilderFactory.EncodingMode.NONE);
restTemplate.setUriTemplateHandler(defaultUriBuilderFactory);
restTemplate.exchange(putUrl.toString(), org.springframework.http.HttpMethod.PUT, entity, Void.class);
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.