AWS DefaultCredentialProvider set by AWS_PROFILE env var not working for terraform remote state

Question

I have a simple Terraform root that provisions some AWS resources. It was initially set up with default local state. I use an AWS Profile to specify the target environment:

$ export AWS_PROFILE="some-aws-profile"
$ aws sts get-caller-identity
{
    "UserId": "REDACTED:REDACTED",
    "Account": "account_id",
    "Arn": "arn:aws:sts::account:assumed-role/somerolename/someusername"
}

And I can run terraform plan or terraform apply - resources get created in the target account. provider "aws" is configured with a region parameter only, all other details / creds are controlled via the AWS_PROFILE env var.

Now I am looking to move state to remote, with an S3 backend.

terraform {
  backend "s3" {
    bucket = "my-bucket-name"
    key    = "some/path/to/terraform.tfstate"
    region = "eu-west-1"
  }
}

When I run terraform init with this, an error is thrown: Error: error configuring S3 Backend: no valid credential sources for S3 Backend found. I have also tried adding profile = "some-aws-profile" into the s3 backend block, but the same still fails.

Does a terraform / backend block use a different credential provider chain? Any reason why this backend config is not able to use AWS_PROFILE implicitly from environment var, or even when profile is added?

I don't have any .credentials files that I use for auth - in my local environment, i am using aws sso login to automatically manage credentials via /cache/ subdirs in ~/.aws/sso or ~/.aws/cli - is this the part that is not compatible with backend?

edit adding in a snippet from ~/.aws/config

This is what my profile looks like:

[profile some-aws-profile]
sso_start_url = https://myhostname.awsapps.com/start/#/
sso_region = eu-west-1
sso_account_id = <actual_account_id>
sso_role_name = somerolename
region = eu-west-1
output = json

To set up auth, i use aws sso login once AWS_PROFILE is set, and I authorize the request for temporary credentials in whereever CLI stores them.

Answer 1

This was not working in 0.13.6 with the latest version of terraform provider aws (4.15.1).

Upgrading to TF 1.2.0 resolved this - SSO profile is used for credential loading in the S3 backend.