Using service principal to access Azure blob storage

Question

I want to access a private blob store, from python, by using credentials from an active directory service principal.

I am aware of this related question How do I authenticate a user against an Azure storage blob in python? This has helped me get this far but now I'm stuck.

I can authenticate and get a token which let me list containers, create new containers, but will not let me list or access any blobs.

I wish to set this up via the az cli.

Service principal has been set up like so:

az ad sp create-for-rbac -n "http://$NAME" --role Contributor \
    --scopes "/subscriptions/$SUB_ID/resourceGroups/$RESOURCE_GROUP" 

Which I believe should give full access but I also added this to be sure:

az role assignment create \
    --role "Storage Blob Data Contributor" \
    --assignee-object-id "$OBJECT_ID" \
    --assignee-principal-type "ServicePrincipal" \
    --scope "/subscriptions/$SUB_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.Storage/storageAccounts/$STORAGE_ACCOUNT/blobServices/default/containers/$CONTAINER"

I am then authenticating like so:

from azure.common.credentials import ServicePrincipalCredentials
import adal
from azure.storage.blob import (
    BlockBlobService,
    ContainerPermissions,
)
from azure.storage.common import (
    TokenCredential
)

# Tenant ID for your Azure Subscription
TENANT_ID = TENANT

# Your Service Principal App ID
CLIENT = APP_ID

# Your Service Principal Password
KEY = PASSWORD

# RESOURCE = "https://storage.azure.com/" # using this resource has the same behaviour as uncommented one. Using no resource fails with authentication errors
RESOURCE = f"https://{ACCOUNT_NAME}.blob.core.windows.net"

credentials = ServicePrincipalCredentials(
    client_id = CLIENT,
    secret = KEY,
    tenant = TENANT_ID,
    resource = RESOURCE
)
tokenCre = TokenCredential(credentials.token["access_token"])

I then try to use the blob service as so

blobService = BlockBlobService(account_name=ACCOUNT_NAME, token_credential=tokenCre)

print ([c.name for c in blobService.list_containers()]) # successfully lists containers
print(blobService.create_container('test')) # prints "True" and the container is created


blobService.list_blobs(CONTAINER_NAME) # fails with AzureHttpError: This request is not authorized to perform this operation using this permission. ErrorCode: AuthorizationPermissionMismatch 

blobService.get_blob_to_bytes("test", "hello.txt") # fails with same error

As shown in the code block above I seem to be able to take 'container' level actions but no 'blob' level actions. Any thing like listing blobs, reading a blob etc gets the error:

AzureHttpError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. ErrorCode: AuthenticationFailed
<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:86ff0241-c01e-00d4-512c-6e22b5000000
Time:2019-09-18T14:20:23.5619727Z</Message><AuthenticationErrorDetail>Audience validation failed. Audience did not match.</AuthenticationErrorDetail></Error>

Any ideas?

Answer 1

If you want to use Azure AD access token to access your Azure storage, you must assign necessary storage ** role to your resource (which you did in your second az command).

However, in your second az command:

az role assignment create \
    --role "Storage Blob Data Contributor" \
    --assignee-object-id "$OBJECT_ID" \
    --assignee-principal-type "ServicePrincipal" \
    --scope "/subscriptions/$SUB_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.Storage/storageAccounts/$STORAGE_ACCOUNT/blobServices/default/containers/$CONTAINER"

You only set the scope to a specific container : $CONTAINER . So, you can only access blobs under that container.

I tested, and got a success.

from azure.common.credentials import ServicePrincipalCredentials
import adal
from azure.storage.blob import (
    BlockBlobService,
    ContainerPermissions,
)
from azure.storage.common import (
    TokenCredential
)

# Tenant ID for your Azure Subscription
TENANT_ID = "e4c9****-****-****-****-230b****57fb"

# Your Service Principal App ID
CLIENT = "3bee****-****-****-****-b0b8****f7a4"

# Your Service Principal Password
KEY = "*******************"

ACCOUNT_NAME = "storagetest789"

CONTAINER_NAME = "newcontainer"

RESOURCE = "https://storage.azure.com/"

credentials = ServicePrincipalCredentials(
    client_id = CLIENT,
    secret = KEY,
    tenant = TENANT_ID,
    resource = RESOURCE
)
tokenCre = TokenCredential(credentials.token["access_token"])
blobService = BlockBlobService(account_name=ACCOUNT_NAME, token_credential=tokenCre)

print("\nList blobs in the container")
generator = blobService.list_blobs(CONTAINER_NAME)
for blob in generator:
    print("\t Blob name: " + blob.name)

print("\nOutput test.txt")
blob = blobService.get_blob_to_text(CONTAINER_NAME, "test.txt")
print(blob.content)

Result:

If I try to access other containers, I will get the same error as yours. But I don't know why the container create operation is allowed. It seems to be an oversight of access control.

If you want to manage the whole storage account, then you need to assign storage account scope to your service principal. Then you can access other containers in that storage account.

Answer 2

With Python library azure-storage-blob 12.2, and azure-identity (latest API as of Jan 2020) (modified version from Jack Lia's answer)

from azure.identity import ClientSecretCredential
from azure.storage.blob import BlobServiceClient

# Tenant ID for your Azure Subscription
TENANT_ID = "e4c9****-****-****-****-230b****57fb"

# Your Service Principal App ID
CLIENT = "3bee****-****-****-****-b0b8****f7a4"

# Your Service Principal Password
KEY = "*******************"

ACCOUNT_NAME = "storagetest789"

CONTAINER_NAME = "newcontainer"

RESOURCE = "https://storage.azure.com/"

credentials = ClientSecretCredential(TENANT_ID, CLIENT, KEY)
blobService = BlobServiceClient(
   "https://{}.blob.core.windows.net".format(ACCOUNT_NAME),
   credential=credentials
)

print("\nList blobs in the container")

container = blobService.get_container_client(CONTAINER_NAME)
for blob in container.list_blobs():
    print("\t Blob name: " + blob.name)

print("\nOutput test.txt")
blob = container.get_blob_client("test.txt")
print(blob.download_blob().readall())