AWS SDK 1.11.844 is throwing Continuously ERROR while connecting to Kinesis Stream

Question

I am Using Spring Cloud Data Stream and Kinesis Spring Binder to connect to Kinesis: https://github.com/spring-cloud/spring-cloud-stream-binder-aws-kinesis/blob/master/spring-cloud-stream-binder-kinesis-docs/src/main/asciidoc/overview.adoc By default, it uses aws-sdk 1.11.415 and which can't be used for Assumed Role while deploying in Kubernetes environment (AWS EKS). I have figured out that I need to upgrade the library to aws-sdk 1.11.844 to support the WebIdentityToken based credential provider. This is working in Kubernetes. Now from my local machine, I am trying to connect to Kinesis by enabling instance profile (disabling the WebIdentityToken based flow), it keeps on throwing below error and not connecting. when I comment this upgraded library it goes back to aws-sdk 1.11.415 which is working as expected. But I need to upgrade this for WebIdentityToken based credential provider.

Please help me if anybody having any solution for this. I can't change to AWS SDKV2 as Spring Binder will not support that. Any other solution will be helpful.

    com.amazonaws.SdkClientException: Failed to connect to service endpoint: 
    at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:100) ~[aws-java-sdk-core-1.11.844.jar:?]
    at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:70) ~[aws-java-sdk-core-1.11.844.jar:?]
    at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:75) ~[aws-java-sdk-core-1.11.844.jar:?]
    at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:66) ~[aws-java-sdk-core-1.11.844.jar:?]
    at com.amazonaws.util.EC2MetadataUtils.getItems(EC2MetadataUtils.java:402) ~[aws-java-sdk-core-1.11.844.jar:?]
    at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:371) ~[aws-java-sdk-core-1.11.844.jar:?]
    at org.springframework.cloud.aws.context.support.env.AwsCloudEnvironmentCheckUtils.isRunningOnCloudEnvironment(AwsCloudEnvironmentCheckUtils.java:38) ~[spring-cloud-aws-context-2.2.2.RELEASE.jar:2.2.2.RELEASE]
......
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) [spring-boot-2.3.1.RELEASE.jar:2.3.1.RELEASE]
    at com.baxter.renal.app.s3.poc.CAPDTreatmentUploadApplication.main(CAPDTreatmentUploadApplication.java:22) [classes/:?]
Caused by: java.net.SocketTimeoutException: connect timed out
    at java.net.PlainSocketImpl.waitForConnect(Native Method) ~[?:?]
    at java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:107) ~[?:?]

Answer 1

I had this problem locally. Turning off ContextInstanceDataAutoConfiguration is the best solution. Just add

spring.autoconfigure.exclude: org.springframework.cloud.aws.autoconfigure.context.ContextInstanceDataAutoConfiguration

to your application.yml file.

Answer 2

The problem is that your local machine doesn't have an instance profile.

If you look at the stack trace, you'll see EC2MetadataUtils.getItems() . This is trying to read from the instance metadata endpoint, http://169.254.169.254 . On an EC2 instance that returns quickly; on your local machine it will time-out because that IP address doesn't exist.

If you're looking to consume AWS services from your local machine, you will have to use a credentials provider that retrieves the information locally. I recommend using DefaultAWSCredentialsProviderChain , which looks for local configuration (either in $HOME/.aws or via environment variables or system properties), and also supports instance profiles (in case you're running on an EC2 instance).