.NET Core worker - since enabling Docker Compose my service can't read IAM credentials from appsettings.json

Question

I am building a small .NET Core worker service to read messages from an AWS SQS queue. To do this, it requires IAM credentials in the form of an access and secret key. I had originally been running this project as a docker container, and it has been working fine. I wanted to enable the ability to run this container multiple times, so multiple instances of the same service. I enabled Docker Compose in Visual Studio and it added the base docker compose YAML file to the project. Now when I try to run the solution in debug, I get the following error:

An exception of type 'Amazon.Runtime.AmazonServiceException' occurred in System.Private.CoreLib.dll but was not handled in user code: 'Unable to get IAM security credentials from EC2 Instance Metadata Service.'

I am assuming this is because somehow now the solution can't read the access and secret key from my appSettings.json but I'm unsure how to fix this.

Here is my DockerFile

FROM mcr.microsoft.com/dotnet/core/runtime:3.1-buster-slim AS base
WORKDIR /app

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-buster AS build
WORKDIR /src
COPY ["WorkerServiceDocker.csproj", ""]
RUN dotnet restore "./WorkerServiceDocker.csproj"
COPY . .
WORKDIR "/src/."
RUN dotnet build "WorkerServiceDocker.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "WorkerServiceDocker.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
RUN mkdir -p /app/logging
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "WorkerServiceDocker.dll"]

Here is the docker compose file created by Visual Studio

version: '3.4'

services:
  workerservicedocker:
    image: ${DOCKER_REGISTRY-}workerservicedocker
    build:
      context: .
      dockerfile: Dockerfile

The method that fails is:

public async void TriggerMessageGetAsync(string queueUrl, int pollingTime)
{
    var _messages = await GetMessagesAsync(queueUrl, pollingTime);
    if (_messages.Count == 0)
    {
        _logger.LogInformation($"No messages found.");
    }
    else
    {
        foreach (var message in _messages)
        {
            _logger.LogInformation($"Message {message.MessageId} received OK");
        }
    }
}

And my configuration is loaded from appSettings.json in the Program.cs

public class Program
{
    public static void Main(string[] args)
    {
        Session.InstanceGuid = Guid.NewGuid().ToString();

        File.Create("init.log").Close();
        var logWriter = new StreamWriter("init.log");
        logWriter.WriteLine("Assembly Path: " + Path.GetDirectoryName(Assembly.GetEntryAssembly().Location));
        logWriter.WriteLine($"Session Guid: {Session.InstanceGuid}");
        logWriter.WriteLine($"Container DateTime: {DateTime.Now}");
        logWriter.Dispose(); 
        CreateHostBuilder(args).Build().Run();
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureServices((hostContext, services) =>
            {
                services.AddHostedService<Worker>();
                services.AddSingleton<ILogger, Logger>(); // Using my own basic wrapper around NLog for the moment, pumped to CloudWatch.
                services.AddAWSService<IAmazonSQS>();
            });
}

When I run in debug using the 'Docker Compose' option, the output indicates that it has found AWS credentials?

  Found AWS options in IConfiguration

AWSSDK: Information: Found AWS options in IConfiguration AWSSDK: Information: Found credentials using the AWS SDK's default credential search

I'm not sure what else to include with this question, but am hoping someone can assist. I am clearly missing some configuration or option that ensures the access and secret key can be read from my appSettings.json file. As I said, this was working fine when I was running it as a single docker container from my DockerFile.

Answer 1

I don't see where you actually instruct AWS to use those credentials, so I will solve that as a likely problem.

First, your appsettings.json should be in this format for aws creds:

"AWS": {        
    "AccessKey": "abc",         
    "SecretKey": "xyz",         
    "Region": "ksjs"    
},

And then try this solution ( explicitly setting environment variables before you initialize the aws service)

https://github.com/aws/aws-sdk-net/issues/499#issuecomment-276142625