Kubectl with Gitlab EKS Cluster Error: You must be logged in to the server (Unauthorized)

Question

I'm trying to run kubectl in a EKS Cluster created from a third-party (Gitlab), and I need to access this cluster with kubectl to install metrics server.

But I can't access this cluster with kubectl. I did everything from the documentation. I have setted the trust relationship in the IAM Role of the cluster like above:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "eks.amazonaws.com",
        "AWS": [
          "arn:aws:sts::123456789012:assumed-role/AzureAD_AWS_Admin/me@mycompany.com",
          "arn:aws:iam::123456789012:root"
        ]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

And I have tested assume-role with the external-id like the command above, and everything run seamlessly!

aws sts assume-role --role-arn arn:aws:iam::123456789012:role/eksServiceRole --external-id 2ffd8b2c3d8edaf02104a081af4b78d82f6e770f --role-session-name my_test

So, I followed these steps to assume role and grant permission to my user (For the step: Add designated_user to the ConfigMap if the cluster_creator is an IAM role ) to use kubectl.

But when I run kubectl get pods I have this message again:

error: You must be logged in to the server (Unauthorized)

Answer 1

I had the same problem when following the Gitlab instructions. This guide helps out:

In the aws-services -> IAM -> Roles -> select the role you defined for gitlab.

Now you could either (1) simply add your user to the trusted entities, or (2) update them only temporarily and edit the aws-auth config.

---

(1) Add another statement with your account ID without the external ID condition:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::[account]:user/username"
      },
      "Action": "sts:AssumeRole"
    },
    {
      // Leave untouched
    }
  ]
}

---

(2) Edit the trusted entities by removing the externalId condition and change it to your user-account id (which you configured aws-cli with):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::[gitlabsaccount]:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "[externalId]"
        }
      }
    }
  ]
}

->

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::[account]:user/username"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Then run the cli-command to reconfigure your kubectl cluster access with this role:

aws --region [region] eks update-kubeconfig --name [cluster-name] --role-arn arn:aws:iam::[account]:role/[role-name]

Lastly, use the permissions to add yourself to the aws-auth config map:

kubectl edit -n kube-system configmap/aws-auth

and then add your account:

apiVersion: v1
data:
  mapRoles: |
    ...
  mapUsers: |
    ---
    - userarn: arn:aws:iam::[account]:user/[username]
      username: [username]
      groups:
      - system:masters
...

Do not forget to change back the gitlab-user for the role and external condition.