使用 Python 的 Azure 服务主体和 storage.blob

Question

I'm trying to authenticate with a service principal through python and then accessing azure.storage.blob

Used to do it with:

NAME = '****'
KEY = '****'
block_blob_service = BlockBlobService(account_name=NAME, account_key=KEY, protocol='https')

But I cant make it work with the service principal:

TENANT_ID = '****'

CLIENT = '****'

KEY_SERVICE = '****'

credentials = ServicePrincipalCredentials(
    client_id = CLIENT,
    secret = KEY_SERVICE,
    tenant = TENANT_ID
    )

I'm a little confused how I pair those 2 and whatever I try it just gives me a timeout when I'm trying to upload a blob.

Answer 1

I don't think Azure Storage Service supports service principal credentials. Actually it only accepts two kinds of credentials currently: shared keys and Shared Access Signature (SAS).

Answer 2

Azure Storage works specifically with account name + key (whether primary or secondary). There is no notion of service principal / AD-based access.

Your first example (setting up the blob service endpoint) with account name + key is the correct way to operate.

Note: As Zhaoxing mentioned, you can also use SAS. But from a programmatic standpoint, assuming you are the storage account owner, that doesn't really buy you much.

The only place Service Principals (and AD in general) comes into play is managing the resource itself (eg the storage account, from a deployment/management/deletion standpoint).

Answer 3

This is highly confusing. Once a service principal (SP) is registered in azure portal with a new secret created...

...assigned Contributor role at the Resource Group level...

...any storage accounts and containers therein inherit this role.

In your application, you can create ServicePrincipalCredentials() and a ClientSecretCredential() from the registered SP...

service_credential = ServicePrincipalCredentials(
    tenant = '<yourTenantID>',
    client_id = '<yourClientID>',
    secret = '<yourClientSecret>'
)

client_credential = ClientSecretCredential(
    '<yourTenantID>',
    '<yourClientID>',
    '<yourClientSecret>'
)

From here, create a ResourceManagementClient() ...

resource_client = ResourceManagementClient(service_credential, subscription_id)

...to list RG's, Resources and Storage Accounts.

for item in resource_client.resource_groups.list():
    print(item.name)

for item in resource_client.resources.list():
    print(item.name + " " + item.type)

for item in resource_client.resources.list_by_resource_group('azureStorage'):
    print(item.name)

BUT...from my research, you cannot list blob containers nor blobs within a given container using ResourceManagementClient() !!. So we move to a BlobServiceClient()

blob_service_client = BlobServiceClient(account_url = url, credential=client_credential)

From here, you can list blob containers...

blob_list = blob_service_client.list_containers()
for blob in blob_list:
    print(blob.name + " " + str(blob.last_modified))

BUT... From my research, you cannot list blobs within a container!!!

container_client = blob_service_client.get_container_client('testcontainer')

blob_list = container_client.list_blobs()
for blob in blob_list:
    print("\t" + blob.name)

---------------------------------------------------------------------------
StorageErrorException                     Traceback (most recent call last)
~/anaconda3_501/lib/python3.6/site-packages/azure/storage/blob/_models.py in _get_next_cb(self, continuation_token)
    599                 cls=return_context_and_deserialized,
--> 600                 use_location=self.location_mode)
    601         except StorageErrorException as error:

~/anaconda3_501/lib/python3.6/site-packages/azure/storage/blob/_generated/operations/_container_operations.py in list_blob_flat_segment(self, prefix, marker, maxresults, include, timeout, request_id, cls, **kwargs)
   1142             map_error(status_code=response.status_code, response=response, error_map=error_map)
-> 1143             raise models.StorageErrorException(response, self._deserialize)
   1144 

StorageErrorException: Operation returned an invalid status 'This request is not authorized to perform this operation using this permission.'

During handling of the above exception, another exception occurred:

HttpResponseError                         Traceback (most recent call last)
<ipython-input-104-7517e7a6a19f> in <module>
      1 container_client = blob_service_client.get_container_client('testcontainer')
      2 blob_list = container_client.list_blobs()
----> 3 for blob in blob_list:
      4     print("\t" + blob.name)

~/anaconda3_501/lib/python3.6/site-packages/azure/core/paging.py in __next__(self)
    120         if self._page_iterator is None:
    121             self._page_iterator = itertools.chain.from_iterable(self.by_page())
--> 122         return next(self._page_iterator)
    123 
    124     next = __next__  # Python 2 compatibility.

~/anaconda3_501/lib/python3.6/site-packages/azure/core/paging.py in __next__(self)
     72             raise StopIteration("End of paging")
     73 
---> 74         self._response = self._get_next(self.continuation_token)
     75         self._did_a_call_already = True
     76 

~/anaconda3_501/lib/python3.6/site-packages/azure/storage/blob/_models.py in _get_next_cb(self, continuation_token)
    600                 use_location=self.location_mode)
    601         except StorageErrorException as error:
--> 602             process_storage_error(error)
    603 
    604     def _extract_data_cb(self, get_next_return):

~/anaconda3_501/lib/python3.6/site-packages/azure/storage/blob/_shared/response_handlers.py in process_storage_error(storage_error)
    145     error.error_code = error_code
    146     error.additional_info = additional_data
--> 147     raise error
    148 
    149 

HttpResponseError: This request is not authorized to perform this operation using this permission.
RequestId:e056fe39-b01e-0007-425c-20a63f000000
Time:2020-05-02T08:32:02.2204809Z
ErrorCode:AuthorizationPermissionMismatch
Error:None

The only way I've found to list blobs within a container (and to do other things like copy blobs, etc), is to create the BlobServiceClient using a Connection String rather than TenantID, ClientID, ClientSecret .

There is some more info about using a token to access blob resources here and here and here , but I havent' been able to test yet.