AWS Cloudwatch logs with Docker Container - NoCredentialProviders: no valid providers in chain

Question

My docker-compose file:

version: '2'
services:
  scraper:
    build: ./Scraper/
    logging:
      driver: "awslogs"
      options:
         awslogs-region: "eu-west-1"
         awslogs-group: "doctors-logs"
         awslogs-stream: "scrapers-stream"
    volumes:
      - ./Scraper/spiders:/spiders

I have added my AWS credentials to my mac using the aws configure command and the credentials are stored correctly in ~/.aws/credentials

When I run docker-compose up I get the following error:

ERROR: for scraper Cannot start service scraper: Failed to initialize logging driver: NoCredentialProviders: no valid providers in chain.

Deprecated. For verbose messaging see aws.Config.CredentialsChainVerboseErrors

ERROR: Encountered errors while bringing up the project.

I believe this is because I need to set the AWS credentials in the Docker Daemon but I cannot work out how this is done on macOs Sierra.

Answer 1

I figured out. When rolling your own EC2 instance (without using automated solutions like Beanstalk), you need to assign a role to your EC2 instance so it will be able to communicate with other AWS services.

The policy is the one that Docker docs provide in https://docs.docker.com/engine/admin/logging/awslogs/

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

then you need to attach this policy to a role

the role is the first one called "Amazon EC2" that reads "Allows EC2 instances to call AWS services on your behalf."

Since you are limiting your access only to CloudWatch, you're good to go. Then, in your EC2 listing, attach the role to your instance using "Attach/Replace IAM Role":

You should be good to go!