Gdal connection to Azure Data Lake Storage (Gen 2) virtual file using AZURE_STORAGE_ACCESS_TOKEN

Question

I want to access my Azure Data Lake Storage (Gen 2) files from gdal version 3.5 using authentication with AZURE_STORAGE_ACCESS_TOKEN as described here: https://gdal.org/user/virtual_file_systems.html#vsiadls

None of the other authentication options be are enabled in our organization storage accounts (AZURE_STORAGE_CONNECTION_STRING, AZURE_NO_SIGN_REQUEST=YES, AZURE_STORAGE_SAS_TOKEN...)

Not sure why oauth2 token call for Azure doesn't work - see see https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-client-creds-grant-flow

url: https://login.microsoftonline.com/<TENANT_ID>/oauth2/token

headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': 'application/json'} data request: {'client_id': '<client_id>', 'resource': 'https://storage.azure.com/.default', 'client_secret': '[REDACTED]', 'grant_type': 'client_credentials'}

Response: {'token_type': 'Bearer', 'expires_in': '3599', 'ext_expires_in': '3599', 'expires_on': '1663774788', 'not_before': '1663770888', 'resource': '<azure_enterprise_app_id>', 'access_token': '<REDACTED>'}

Answer 1

Turns out you can set AZURE_STORAGE_ACCESS_TOKEN using MSAL (python SDK) see https://github.com/AzureAD/microsoft-authentication-library-for-python

This works:

from msal import ConfidentialClientApplication


def get_token():
    app = ConfidentialClientApplication(
        os.getenv("AZURE_SP_CLIENT_ID"),
        authority="https://login.microsoftonline.com/mmcglobal.onmicrosoft.com",
        client_credential=os.getenv("AZURE_SP_CLIENT_SECRET"),
    )

    result = app.acquire_token_for_client(scopes="https://storage.azure.com/.default")

    if "access_token" in result:
        # Call a protected API with the access token.
    #     print(result["token_type"])
        print("Setting access token")
    else:
        print(result.get("error"))
        print(result.get("error_description"))
        print(result.get("correlation_id"))  # You might need this when reporting a bug.
    return result['access_token']


os.environ["AZURE_STORAGE_ACCOUNT"] = <account_name>
os.environ["AZURE_STORAGE_ACCESS_TOKEN"]=get_token()

Now I can load files from /vsiadls/