AWS DotNet SDK 错误：无法从 EC2 实例元数据服务获取 IAM 安全凭证

Question

I use an example from here in order to retreive a secret from AWS SecretsManager in c# code.

I have set credentials locally via AWS CLI, and I am able to retreive secret list using AWS CLI command "aws secretsmanager list-secrets".

But c# console app fails with an error:

> Unhandled exception. System.AggregateException: One or more errors occurred. (Unable to get IAM security credentials from EC2 Instance Metadata Service.)
 ---> Amazon.Runtime.AmazonServiceException: Unable to get IAM security credentials from EC2 Instance Metadata Service.
   at Amazon.Runtime.DefaultInstanceProfileAWSCredentials.FetchCredentials()
   at Amazon.Runtime.DefaultInstanceProfileAWSCredentials.GetCredentials()
   at Amazon.Runtime.DefaultInstanceProfileAWSCredentials.GetCredentialsAsync()
   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at System.Threading.Tasks.Task`1.get_Result()
   at AWSConsoleApp2.GetSecretValueFirst.GetSecret() in D:\Work\Projects\Training\AWSConsoleApp2\AWSConsoleApp2\GetSecretValueFirst.cs:line 53
   at AWSConsoleApp2.Program.Main(String[] args) in D:\Work\Projects\Training\AWSConsoleApp2\AWSConsoleApp2\Program.cs:line 11

When I change original constructor call

IAmazonSecretsManager client = new AmazonSecretsManagerClient();

with adding inherited parameter of type AWSCredentials

IAmazonSecretsManager client = new AmazonSecretsManagerClient(new StoredProfileAWSCredentials());

it works fine.

Class StoredProfileAWSCredentials is obsolete but it works to use it. I use libraries that work without errors on the other machines and I cannot change them.

I use credentials for user that belongs to Administrators group and has full access to SecretsMnager. Region has set properly in c# code, profile is default.

Any ideas? Thanks for advance

Answer 1

I had the same issue, here is how I fixed it on my development environment

1. I created an AWS profile using the AWS extension for Visual studio
2. Once the profile is set up the credentials are passed using the profile and it worked fine for me

Point to note here, the user profile accessing the key manager should have a valid security group assigned for the Secrets manager.

Try it out let me know, how it went.

Answer 2

Same issue and resolved by deleting $HOME/.aws/config and credentials files and recreating with AWS CLI.

In my case I was switching laptops from Windows to a new MBP. I had setup my new environment by copying the .aws directory files and confirmed that AWS CLI worked correctly. Confusingly the dotnet SDK failed with same errors.

Answer 3

使用 AWS 提供的数据运行以下命令并按照提示操作：

aws configure

Answer 4

I've run into this issue a number of times, but have not been able to resolve it using the above solutions.

What has worked for me is explicitly setting my AWS profile using the AWS_PROFILE environment variable and setting it to the profile I want to use.

Today I ran into this issue again, where even that didn't work. What eventually solved it was setting the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.

I dread the day where I run out of alternative ways to supply credentials to AWS.

Answer 5

我有同样的问题，并通过将 Visual Studio 中的 AWS 配置文件的名称更改为默认值来解决它。

Answer 6

The question is not exactly my problem, but it's the first hit on google so I figured I'd chip in just in case.

I got the exact above error when issuing

dotnet lambda list-layers

It seems like the dotnet cli uses the AWS_PROFILE variable and does not default to AWS_DEFAULT_PROFILE . In my company, the AWS_DEFAULT_PROFILE is mapped to an identity provider, thus I do not manage different access with different profiles and the default profile is empty. As a workaround, run your command like this

AWS_PROFILE=$AWS_DEFAULT_PROFILE dotnet lambda list-layers

This way the CLI will use the correct credentials.

Answer 7

make sure you have the latest version of EC2config installed https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/UsingConfig_Install.html

Thanks

Answer 8

Just add env variables in control panel AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. The actual value of either is not important. I have set them both to a space (' '). Don't know why it works but it works. It does seem to take longer to log in. It seems that instead of going to the buggy flow, the SDK tries to use the env vars, fails and about after 30 seconds or so logs in as required.

Tested it on two different Win10 PCs with no AWS CLI installed or any AWS profile configured. The issue was recreated 100% and the described w/a fixed it.

Answer 9

I had the same issue and it turned out to be because I had AWS_ACCESS_KEY_ID , AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN uppercased in my credentials file. Changing the keys to lowercase solved it for me.

Answer 10

Since AWS SDK credentials configuration is causing a lot of headache, I'll throw in some context. First of all, if you are using dotnet core, use the AWSSDK.Extensions.NETCore.Setup package ( https://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/net-dg-config-netcore.html ), which will respect your appsettings.json.

{
  "AWS": {
    "Region": "eu-west-1",
    "Profile": "theprofileyouwantouse"
  }
}

csproj:

  <ItemGroup>
    <PackageReference Include="AWSSDK.Extensions.NETCore.Setup" Version="3.7.1" />
    <PackageReference Include="AWSSDK.SecurityToken" Version="3.7.1.71" />
  </ItemGroup>

Example:

var config = host.Services.GetService<IConfiguration>();
var options = config.GetAWSOptions();
using var client = options.CreateServiceClient<IAmazonSecurityTokenService>();
var result = await client.GetCallerIdentityAsync(new Amazon.SecurityToken.Model.GetCallerIdentityRequest { });

This will try to pick up encrypted credentials in ~/AppData/Local/AWSToolkit and secondly based on your shared config file (~/.aws/config). As of november 2021, it does not utilize aws_access_key_id, aws_secret_access_key, aws_session_token in the version 1 shared credentials file (~/.aws/credentials)*

Next, if the roles you are assuming are AWS SSO, you need the following packages in your csproj file:

    <PackageReference Include="AWSSDK.SSO" Version="3.7.0.94" />
    <PackageReference Include="AWSSDK.SSOOIDC" Version="3.7.0.94" />

*If you happen to have invertedly added your credentials to your shared credentials file (~/.aws/credentials) as [profile myprofile] instead of just [myprofile] the SDK will not behave as you expected, so delete that. If your credentials file is fine, then you don't have to touch it, but keep in mind that the SDK will noe use the cached credentials if any found in that file.

Now, the author does not use the AWSSDK.Extensions.NETCore.Setup package, which means that we are getting a slightly different credentials resolving path. Most importantly: appsettings.json is not respected, this means you must specify the profile you want to use differently, for example by using the AWS_PROFILE environment variable.

Secondly, we are landing directly in the FallbackCredentialsFactory.cs which does this when resolving credentials:

            CredentialsGenerators = new List<CredentialsGenerator>
            {
#if BCL
                () => new AppConfigAWSCredentials(),            // Test explicit keys/profile name first.
#endif
                () => AssumeRoleWithWebIdentityCredentials.FromEnvironmentVariables(),
                // Attempt to load the default profile.  It could be Basic, Session, AssumeRole, or SAML.
                () => GetAWSCredentials(credentialProfileChain),
                () => new EnvironmentVariablesAWSCredentials(), // Look for credentials set in environment vars.
                () => ECSEC2CredentialsWrapper(proxy),      // either get ECS credentials or instance profile credentials
            };

Now the last step in resolving credentials "ECSEC2" has a fallback which returns this:

DefaultInstanceProfileAWSCredentials.Instance

Which leads us to the error which the author sees.

Summary:

1. If you are not using AWSSDK.Extensions.NETCore.Setup, specify the profile using an ENV-variable in launch.json or launchSettings.json if you are going to use the default constructor like the author
2. Rember to add the AWS SSO packages if needed

Answer 11

In the project defaults.json, verify the profile value. in my case it was empty "profile": "" . After setting the profile name, was able to publish

Answer 12

I was deploying to a dot net core web application to an on prem server over IIS and had the same exact issue. No matter what I did the application would not recognize my credentials configured via AWS CLI (aws configure).

I ended up setting AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with my keys via the Windows environment variables and restarting the server.

The following article was very helpful in understanding the AWS SDK credential loading Client Factory https://www.stevejgordon.co.uk/credential-loading-and-the-aws-sdk-for-dotnet-deep-dive

Answer 13

In my case the config and credentials files were set up correctly in C:/Users//.aws folder so they should have been found by default. However, on a previous project I had set up different credentials (no longer valid) in C:/Users//AppData/Local/AWSToolkit referred to in AWS documentation as the AWS SDK Store. The SDK store is always checked first and then falls back to the default user credentials file. See the following: https://aws.amazon.com/blogs/developer/referencing-credentials-using-profiles/ . The simplest solution in my case was simply to delete the files in the AWSToolkit folder. As an alternative I could have set up the SDK Store correctly.

Answer 14

I had the same problem in .NET core 5 with AWS and I solved it by :

what I had :

I had config and credentials files in C:\Users\.aws.

In StartUp.cs after initialized AWS options I added:

#if Debuge
   options.Profile="default";
   options.ProfileLocations="C:\\Users\\.aws\\credentials";
#endif

Answer 15

If anyone is using docker-compose and getting this error, I added this to my docker-compose.override.yml file and it was able to read my credentials

volumes:
  - ~/.aws/:/root/.aws:ro

Answer 16

I had the same issue - Amazon.Runtime.AmazonServiceException: 'Unable to get IAM security credentials from EC2 Instance Metadata Service.'

I was working with Dot Net Core Microservice, I got this error.

Solution - I removed the AWS credentials path which was mentioned in all the different setting files like appsettings.Debug.json and appsettings.Development.json.

This AWS credentials path should be mentioned in only the file appsettings.json. remove from all other files.

Answer 17

Posting this here as it is at least the 2nd time I've caused the error via a self-inflicted misconfiguration.

VS 2022
AWS Toolkit 1.38.0.0

None of the existing answers worked, but they did point to a few configuration issues. One or more of these configuration settings did/could cause the error:

AWS Toolkit Explorer

When using the AWS Toolkit Explorer to define the profile, make sure the correct profile is selected.

Personally, I no longer define a default profile. I use named profiles for all credentials. This is especially useful when dealing with multiple profiles (I have at least 10+).

Environmental Variables

I've found the most consistent way to run an AWS process locally (or as a service) is to set the AWS_PROFILE Environmental Variable. Example:

Environment.SetEnvironmentVariable( "AWS_PROFILE", "<profle_name>", EnvironmentVariableTarget.Process );

In this particular case, I used an old profile name that had been changed. In other words, if you're using an Environmental Variable to set the AWS_PROFILE , make sure the profile name is correct.

When using the Environmental Variable approach, do not set an Environmental Variable for AWS_REGION . The AWS_PROFILE defines the region when the profile is correctly defined.

Other Thoughts...

When configured correctly, there should be no need for any other AWS environmental variable. Not even empty variables for:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN