How to create service principal in azure using azuread_application in terraform, Error Insufficient privileges to complete the operation

Question

Trying to create a service principal in Terraform to be the service principal in the cluster I create in another file. But on creation the failure comes with "azuread_application" "aks_sp" in the file below

data "azurerm_subscription" "current" {}

data "azurerm_client_config" "default" {}
data "azuread_client_config" "defualt" {} # added with azuread 0.8.0

data "azurerm_role_definition" "builtin" {
  name = "Contributor"
}
# Resource group is needed to manage users priveleges
resource "azurerm_resource_group" "test_rs" {
  name      = var.resource_group_name # mwk8ResourceGroup
  location  = var.location # eastus2
}

# Service principal for cluster
# first you need an azure application
resource "azuread_application" "aks_sp" {
  name                        = var.clustername
  homepage                    = "https://${var.clustername}"
  identifier_uris             = ["https://${var.clustername}"]
  reply_urls                  = ["https://${var.clustername}"]
  available_to_other_tenants  = false # default
  public_client               = false # default
  oauth2_allow_implicit_flow  = false # default
}

# service principal
resource "azuread_service_principal" "sp" {
  application_id                = azuread_application.aks_sp.application_id
  app_role_assignment_required  = false # default
}

# create random password
resource "random_password" "aks_rnd_sp_pwd" {
  length  = 16
  special = true
}

resource "azuread_service_principal_password" "aks_sp_pwd" {
  service_principal_id  = azuread_service_principal.sp.id
  value                 = random_password.aks_rnd_sp_pwd.result
  end_date              = "2099-01-01T01:01:01Z"
}

resource "azurerm_role_assignment" "aks_sp_role_assignment" {
  scope                = data.azurerm_subscription.current.id
  role_definition_name = data.azurerm_role_definition.builtin.name
  principal_id         = azuread_service_principal.sp.id

  depends_on = [
    azuread_service_principal_password.aks_sp_pwd
  ]
}

I need the service principal on the cluster because the cluster later is filled by kubernetes with and kubernetes needs the service principal to have the role of contributor for the dynamic allocation of a disk. I have gotten all of this to work if I manually create the cluster and the service principal then release the cluster. But I am working to have this all done by terraform which is failing with this error.

Error: graphrbac.ApplicationsClient#Create: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="Unknown" Message="Unknown service error" Details=[{"odata.error":{"code":"Authorization_RequestDenied","date":"2020-04-29T16:27:42","message":{"lang":"en","value":"Insufficient privileges to complete the operation."},"requestId":"hash"}}]

I have added the subscription_id , client_id , client_secret , and tennant_id to the provider "azuread" block but this doesn't seem to solve the issue.

All the issues I found related to this on github all seem to have dead links to documentation from both microsoft and terraform.

github similar issue

I have already created the rbac role of Contributor for the subscription and added that azuread provider as stated before. Are there steps I am missing? Are there more cli authentication steps I need to complete? I am working through a local powershell authed to azure with the cli. Also any updated documentation links would help as well.

EDIT 1: Provider block

provider "azurerm" {
  version = "~>2.7" # https://github.com/terraform-providers/terraform-provider-azurerm/blob/master/CHANGELOG.md
  features {}
}

provider "azuread" {
  version = "~>0.8.0" # https://github.com/terraform-providers/terraform-provider-azuread/blob/master/CHANGELOG.md

  subscription_id = "subscription_id"
  client_id       = "app_id" # "appId"
  client_secret   = "password"
  tenant_id       = "tennant_id" # "appOwnerTenantId"
}

provider "kubernetes" {
  load_config_file        = false
  host                    = azurerm_kubernetes_cluster.cluster_1.kube_config.0.host
  username                = azurerm_kubernetes_cluster.cluster_1.kube_config.0.username
  password                = azurerm_kubernetes_cluster.cluster_1.kube_config.0.password
  client_certificate      = base64decode(azurerm_kubernetes_cluster.cluster_1.kube_config.0.client_certificate)
  client_key              = base64decode(azurerm_kubernetes_cluster.cluster_1.kube_config.0.client_key)
  cluster_ca_certificate  = base64decode(azurerm_kubernetes_cluster.cluster_1.kube_config.0.cluster_ca_certificate)
}

Answer 1

Terraform uses the old API. You need to add the Application.ReadWrite.Owned but from Azure Active Directory Graph (beware that the permission with same name is also in Microsoft.Graph but that won't work for terraform). Terraform documentation does not mention this unfortunately.

Also you need an Admin consent.

Answer 2

Refer to terraform document , note that

If you're authenticating using a Service Principal then it must have permissions to both Read and write owned by applications and Sign in and read user profile within the Windows Azure Active Directory API.

You could ensure that the service principal that you're using authenticate to Azure ad provider have sufficient privileges to complete the operation. Refer to https://github.com/terraform-providers/terraform-provider-azuread/issues/35#issuecomment-479116655

In this case, you could refer to authenticate to Azure using a Service Principal with a Client Secret for more details. You can create the Service Principal which will have permissions to manage resources in the specified Subscription using the following command:

az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/SUBSCRIPTION_ID"

For me, I just keep the provider "azurerm" block and remove the provider "azuread" block, it will work. For more references: Creating a Kubernetes Cluster with AKS and Terraform