[英]Getting Crc32MismatchException when using dynamodb enhanced client
Describe the bug描述错误
We tried to getItem using DynamoDbEnhancedClient but we got Crc32MismatchException.我们尝试使用 DynamoDbEnhancedClient 获取 Item,但我们得到了 Crc32MismatchException。
Expected Behavior预期行为
I should be able to getItem我应该能够得到Item
Current Behavior当前行为
getting error:得到错误:
software.amazon.awssdk.core.exception.Crc32MismatchException: Expected 1657156166 as the Crc32 checksum but the actual calculated checksum was 3693931191 at software.amazon.awssdk.core.exception.Crc32MismatchException$BuilderImpl.build(Crc32MismatchException.java:88) at software.amazon.awssdk.core.internal.util.Crc32ChecksumValidatingInputStream.validateChecksum(Crc32ChecksumValidatingInputStream.java:62) at software.amazon.awssdk.core.internal.util.Crc32ChecksumValidatingInputStream.close(Crc32ChecksumValidatingInputStream.java:50) at java.base/java.io.FilterInputStream.close(Unknown Source) at software.amazon.awssdk.utils.FunctionalUtils.lambda$safeRunnable$5(FunctionalUtils.java:124) at software.amazon.awssdk.utils.FunctionalUtils.invokeSafely(FunctionalUtils.java:140) at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.lambda$handle$4(JsonResponseHandler.java:94)
software.amazon.awssdk.core.exception.Crc32MismatchException:预期 1657156166 作为 Crc32 校验和,但实际计算的校验和为 3693931191 .amazon.awssdk.core.internal.util.Crc32ChecksumValidatingInputStream.validateChecksum(Crc32ChecksumValidatingInputStream.java:62) at software.amazon.awssdk.core.internal.util.Crc32ChecksumValidatingInputStream.close(Crc32ChecksumValidatingInputStream.java:50) at java.base/java .io.FilterInputStream.close(未知来源)在 software.amazon.awssdk.utils.FunctionalUtils.lambda$safeRunnable$5(FunctionalUtils.java:124)在 software.amazon.awssdk.utils.FunctionalUtils.invokeSafely(8.8823015:42304:124) ) 在 software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.lambda$handle$4(JsonResponseHandler.java:94) at java.base/java.util.Optional.ifPresent(Unknown Source) at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.handle(JsonResponseHandler.java:94) at software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.handle(JsonResponseHandler.java:36) at software.amazon.awssdk.protocols.json.internal.unmarshall.AwsJsonResponseHandler.handle(AwsJsonResponseHandler.java:44) at software.amazon.awssdk.core.http.MetricCollectingHttpResponseHandler.lambda$handle$0(MetricCollectingHttpResponseHandler.java:52) at software.amazon.awssdk.core.internal.util.MetricUtils.measureDurationUnsafe(MetricUtils.java:64) at software.amazon.awssdk.core.http.MetricCollectingHttpResponseHandler.handle(MetricCollectingHttpResponseHandler.java:52) at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(
在 java.base/java.util.Optional.ifPresent(未知来源) 在 software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.handle(JsonResponseHandler.java:94) 在 software.amazon.amazon.awssd json.internal.unmarshall.JsonResponseHandler.handle(JsonResponseHandler.java:36) at software.amazon.awssdk.protocols.json.internal.unmarshall.AwsJsonResponseHandler.handle(AwsJsonResponseHandler.java:44) at software.amazon.awssdk.core. http.MetricCollectingHttpResponseHandler.lambda$handle$0(MetricCollectingHttpResponseHandler.java:52) at software.amazon.awssdk.core.internal.util.MetricUtils.measureDurationUnsafe(MetricUtils.java:64) at software.amazon.awssdk.core.http.MetricCollectingHttpResponseHandler .handle(MetricCollectingHttpResponseHandler.java:52) 在 software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle( AwsSyncClientHandler.java:94) at software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$resultTransformationResponseHandler$7(BaseClientHandler.java:287) at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleSuccessResponse(CombinedResponseHandler.java:97) at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:72) at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:59) at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:40) at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40) at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:
AwsSyncClientHandler.java:94) at software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$resultTransformationResponseHandler$7(BaseClientHandler.java:287) at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleSuccessResponse(CombinedResponseHandler .java:97) at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:72) at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:59 )atsofate.amazon.awssdk.core.interner.http.combinedResponseHandler.handle(commenderedResponsePonseHandler.8824694588888888888888888888888888888888888888.40) ) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java: 30) at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73) at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42) at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78) at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40) at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:50) at sof
30) at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute( ApiCallAttemptTimeoutTrackingStage.java:73) at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42) at software.amazon.awssdk.core.internal.http.pipeline.stages. TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78) at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40) at software.amazon.awssdk.core.internal.http. pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:50) 在 sof tware.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:36) at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:80) at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36) at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56) at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36) at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80) at so
tware.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:36) at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java :80) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36) 在 software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilderPepcuteComposingRequestRequest (RequestPipelineBuilder.java:206) at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56) at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java :36) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80) 处ftware.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60) at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42) at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:48) at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:31) at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) at software.amazon.awssdk.core.internal.882541227
ftware.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60) at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java :42) 在 software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:48) 在 software.amazon.awssdk.core.internal.88254122744.piCutricutrictage.pipecutrictageC288.piCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:48) (ApiCallMetricCollectionStage.java:31) at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder $ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) 在 software.amazon.awssdk.core.internal.882541227 44288.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37) at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26) at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:193) at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:135) at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:161) at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:114) at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:169) at software.amazon.awssdk.core.inter
44288.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37) at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26) at software.amazon.awssdk. core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:193) at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:135) at software.amazon.awssdk.core. internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:161) 在 software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:114) 在 software.amazon.awssdk.core.internal .handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:169) 在 software.amazon.awssdk.core.inter nal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:95) at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45) at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55) at software.amazon.awssdk.services.dynamodb.DefaultDynamoDbClient.getItem(DefaultDynamoDbClient.java:3107) at software.amazon.awssdk.enhanced.dynamodb.internal.operations.CommonOperation.execute(CommonOperation.java:114) at software.amazon.awssdk.enhanced.dynamodb.internal.operations.TableOperation.executeOnPrimaryIndex(TableOperation.java:59) at software.amazon.awssdk.enhanced.dynamodb.internal.client.DefaultDynamoDbTable.getItem(DefaultDynamoDbTable.java:139) at software.amazon.awssdk.enhanced.dynamodb.internal.client.DefaultDynamoDbTable.getItem(DefaultDynamoDbTable.java:146) at software.amazon.awssdk.enhanced.dynamodb.inte
nal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:95) 在 software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45) 在 software.amazon.awssdk.awscore.client.handler AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55) 在 software.amazon.awssdk.services.dynamodb.DefaultDynamoDbClient.getItem(DefaultDynamoDbClient.java:3107) 在 software.amazon.awssdk.Operation.enbhanced.dynamodb. CommonOperation.java:114) 在 software.amazon.awssdk.enhanced.dynamodb.internal.operations.TableOperation.executeOnPrimaryIndex(TableOperation.java:59) 在 software.amazon.awssdk.enhanced.dynamodb.internal.client.DefaultDynamoDbTable.getItem( DefaultDynamoDbTable.java:139) 在 software.amazon.awssdk.enhanced.dynamodb.internal.client.DefaultDynamoDbTable.getItem(DefaultDynamoDbTable.java:146) 在 software.amazon.awssdk.enhanced.dynamodb.inte rnal.client.DefaultDynamoDbTable.getItem(DefaultDynamoDbTable.java:151)
rnal.client.DefaultDynamoDbTable.getItem(DefaultDynamoDbTable.java:151)
Reproduction Steps复制步骤
// https://www.http4k.org/api/org.http4k.client/-ok-http/
val httpClient: HttpHandler = OkHttp()
val awsHttpClient = AwsSdkClient(httpClient)
val dynamoDbClient: DynamoDbClient =
DynamoDbClient.builder()
.region(Region.of(environment.getAwsRegion()))
.httpClient(awsHttpClient)
.build()
val enhancedClient: DynamoDbEnhancedClient =
DynamoDbEnhancedClient.builder()
.dynamoDbClient(dynamoDbClient)
.build()
val tableName = ...
val schema = ...
val table = enhancedClient.table(tableName, schema)
val key = ..
table.getItem(key)
Possible Solution可能的解决方案
This has happened in aws-sdk-java, maybe it is missed in v2?这在 aws-sdk-java 中已经发生,也许它在 v2 中被遗漏了? aws/aws-sdk-java#1018
aws/aws-sdk-java#1018
AWS Java SDK version used AWS Java SDK 使用的版本
2.17.27 2.17.27
JDK version used使用的 JDK 版本
java 11 java 11
Operating System and version操作系统和版本
Amazon Linux 2亚马逊 Linux 2
Reported to github/aws/aws-sdk-java-v2报告到github/aws/aws-sdk-java-v2
Related but not the same stackoverflow question 相关但不相同的stackoverflow问题
If you are using Okhttp as client, you can add a custom header configuration: Accept-Encoding: gzip如果您使用 Okhttp 作为客户端,您可以添加自定义 header 配置:Accept-Encoding: gzip
You can do something like this in kotlin:你可以在 kotlin 中做这样的事情:
val httpClient: HttpHandler = OkHttp()
val awsHttpClient = AwsSdkClient(httpClient)
val dynamoDbClient: DynamoDbClient =
DynamoDbClient.builder()
.region(Region.of(environment.getAwsRegion()))
.httpClient(awsHttpClient)
.overrideConfiguration { o -> o.putHeader("Accept-Encoding", "gzip") }
.build()
val enhancedClient: DynamoDbEnhancedClient =
DynamoDbEnhancedClient.builder()
.dynamoDbClient(dynamoDbClient)
.build()
val tableName = ...
val schema = ...
val table = enhancedClient.table(tableName, schema)
val key = ..
table.getItem(key)
I have not seen others use OkHttp for their AWS clients before, and I believe that this HTTP client uses gzip compression by default.我之前没有看到其他人为他们的 AWS 客户端使用 OkHttp,我相信这个 HTTP 客户端默认使用 gzip 压缩。 To change the configuration you would need to write a.network interceptor as described here .
要更改配置,您需要按照此处所述编写一个.network 拦截器。
I suggest first building with the default HTTP client, which just requires you to remove 2 lines of code.我建议首先使用默认的 HTTP 客户端进行构建,这只需要您删除 2 行代码。 That will allow you to understand if OkHTTP is in fact the issue.
这将使您了解 OkHTTP 是否确实是问题所在。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.