Azure BLOB PUT Rest api - Not able to authenticate with Python

Question

I'm trying to create a Blob in an Azure Container by using Python and the Azure Blob Rest API. it has been an interesting exercise as it's my first time interacting with Azure Rest APIs. I've read the MS documentation about it and also read many questions in this site and according to them my code seems correct however I am not able to make a successful PUT yet. I'm able to execute GET requests (list containers/blobs).

Following the code:

import requests
import datetime
import hmac
import hashlib
import base64

storage_account_name = '<mystorageaccount>'
storage_account_key = '<mystoragekey>'
container_name='test'
api_version = '2015-02-21'
request_time = datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT')

string_params = {
    'verb': 'PUT',
    'Content-Encoding': '',
    'Content-Language': '',
    'Content-Length': '11',
    'Content-MD5': '',
    'Content-Type': 'text/plain; charset=UTF-8',
    'Date': '',
    'If-Modified-Since': '',
    'If-Match': '',
    'If-None-Match': '',
    'If-Unmodified-Since': '',
    'Range': '',
    'CanonicalizedHeaders': 'x-ms-blob-type:BlockBlob' + '\nx-ms-date:' + request_time + '\nx-ms-version:' + api_version,
    'CanonicalizedResource': '/' + storage_account_name +'/'+container_name+ '/' +'fname'
}

string_to_sign = (string_params['verb'] + '\n' 
                  + string_params['Content-Encoding'] + '\n'
                  + string_params['Content-Language'] + '\n'
                  + string_params['Content-Length'] + '\n'
                  + string_params['Content-MD5'] + '\n' 
                  + string_params['Content-Type'] + '\n' 
                  + string_params['Date'] + '\n' 
                  + string_params['If-Modified-Since'] + '\n'
                  + string_params['If-Match'] + '\n'
                  + string_params['If-None-Match'] + '\n'
                  + string_params['If-Unmodified-Since'] + '\n'
                  + string_params['Range'] + '\n'
                  + string_params['CanonicalizedHeaders']
                  + string_params['CanonicalizedResource'])

signed_string = base64.b64encode(hmac.new(base64.b64decode(storage_account_key), msg=string_to_sign.encode('utf-8'), digestmod=hashlib.sha256).digest()).decode()

headers = {
    'x-ms-version' : api_version,
    'x-ms-date' : request_time,
    'x-ms-blob-type': 'BlockBlob',
    'Content-Length': '11',
    'Content-Type': "text/plain; charset=UTF-8",
    'Authorization' : ('SharedKey ' + storage_account_name + ':' + signed_string)
}

url = ('https://' + storage_account_name + '.blob.core.windows.net/'+container_name+'/fname')

r = requests.put(url, headers = headers,data='hello world')
print(r.status_code)
print('\n\n'+r.text)

That's the return error message I got not matter what:

<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:cbf12c65-c01e-00fc-1069-3a41a7000000
Time:2020-06-04T12:11:03.4295368Z</Message><AuthenticationErrorDetail>The MAC signature found in the HTTP request 'c9n6EKq9p6skUs17qGv/bW0yGRGjMzMrP7bgDwjRABg=' is not the same as any computed signature. Server used following string to sign: 'PUT


11

text/plain; charset=UTF-8






x-ms-blob-type:BlockBlob
x-ms-date:Thu, 04 Jun 2020 12:11:02 GMT
x-ms-version:2015-02-21
/<mystorageaccount>/test/fname'.</AuthenticationErrorDetail></Error>

Can someone pls help me understand what I missing here?

Answer 1

You need to include text/plain; charset=UTF-8 text/plain; charset=UTF-8 for Content-Type and 11 for Content-Length in string_params . So your revised code ( string_params variable only) would be something like:

string_params = {
    'verb': 'PUT',
    'Content-Encoding': '',
    'Content-Language': '',
    'Content-Length': '11',
    'Content-MD5': '',
    'Content-Type': 'text/plain; charset=UTF-8',
    'Date': '',
    'If-Modified-Since': '',
    'If-Match': '',
    'If-None-Match': '',
    'If-Unmodified-Since': '',
    'Range': '',
    'CanonicalizedHeaders': 'x-ms-blob-type:BlockBlob' + '\nx-ms-date:' + request_time + '\nx-ms-version:' + api_version,
    'CanonicalizedResource': '/' + storage_account_name +'/'+container_name+ '/' +'fname'
}

Rest of your code looks fine.

UPDATE

You're missing a new line character at the end of CanonicalizedHeaders . So your string_params will be like:

string_params = {
    'verb': 'PUT',
    'Content-Encoding': '',
    'Content-Language': '',
    'Content-Length': '11',
    'Content-MD5': '',
    'Content-Type': 'text/plain; charset=UTF-8',
    'Date': '',
    'If-Modified-Since': '',
    'If-Match': '',
    'If-None-Match': '',
    'If-Unmodified-Since': '',
    'Range': '',
    'CanonicalizedHeaders': 'x-ms-blob-type:BlockBlob' + '\nx-ms-date:' + request_time + '\nx-ms-version:' + api_version + '\n',
    'CanonicalizedResource': '/' + storage_account_name +'/'+container_name+ '/' +'fname'
}

Answer 2

Here is a simpler version of the code with more code reuse:

def generate_key(verb, headers, content_type, storage_account_key, canonicalized_resource):
    string_to_sign = (verb + '\n'
                      + '\n' # Content-MD5
                      + content_type+ '\n'
                      + '\n' #Date
                      + "\n".join([k + ":" + v for k,v in headers.items()]) + '\n'
                      + canonicalized_resource)

    return base64.b64encode(
        hmac.new(base64.b64decode(storage_account_key), msg=string_to_sign.encode('utf-8'),
                 digestmod=hashlib.sha256).digest()).decode()

def request_with_headers(verb, storage_account_name, storage_account_key, container_name, blob_name, data = None):
    api_version = '2018-11-09'
    request_time = datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT')
    headers = {"x-ms-blob-type": "BlockBlob", "x-ms-date": request_time, "x-ms-version": api_version}
    canonicalized_resource = '/' + storage_account_name + '/' + container_name + '/' + blob_name
    content_type = "text/plain; charset=UTF-8"
    sas_token = generate_key(verb, headers, content_type, storage_account_key, canonicalized_resource)

    headers = dict(headers, **{"Content-Type": content_type, "Authorization": "SharedKeyLite %s:%s" % (storage_account_name, sas_token)})
    url = "https://%s.blob.core.windows.net/%s/%s" % (storage_account_name, container_name, blob_name)
    return requests.request(verb, url, headers=headers, data = data)