使用 Bash/Curl 從 Azure Blob Storage 下載文件

Question

我正在嘗試使用以下腳本從 Azure Blob 存儲下載文件：

authorization="SharedKey"

HTTP_METHOD="GET"
request_date=$(TZ=GMT date "+%a, %d %h %Y %H:%M:%S %Z")
storage_service_version="2009-09-19"

# HTTP Request headers
x_ms_date_h="x-ms-date:$request_date"
x_ms_version_h="x-ms-version:$storage_service_version"
x_ms_blob_type_h="x-ms-blob-type:BlockBlob"


# Build the signature string
canonicalized_headers="$${x_ms_date_h}\n$${x_ms_version_h}"
canonicalized_resource="/${STORAGE_ACCOUNT}/${STORAGE_CONTAINER}"

string_to_sign="$${HTTP_METHOD}\n\n\n\n\n\n\n\n\n\n\n\n$${x_ms_blob_type_h}\n$${canonicalized_headers}\n$${canonicalized_resource}"

# Decode the Base64 encoded access key, convert to Hex.

decoded_hex_key="$(echo -n ${STORAGE_KEY} | base64 -d -w0 | xxd -p -c256 | tr -d ' ')"

# Create the HMAC signature for the Authorization header
signature=$(printf "$string_to_sign" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$decoded_hex_key" -binary | base64 -w0)

authorization_header="Authorization: $authorization $STORAGE_ACCOUNT:$signature"
FILE_TYPE="application/x-yml"
DOWNLOAD_FILE="https://${STORAGE_ACCOUNT}.blob.core.windows.net/${STORAGE_CONTAINER}/${FILENAME}"

curl -H "$x_ms_date_h" \
     -H "$x_ms_version_h" \
     -H "$x_ms_blob_type_h" \
     -H "$authorization_header" \
     -H "Content-Type: $${FILE_TYPE}" \
     -f $${DOWNLOAD_FILE} -o ${FILENAME} 

我還使用 Terraform 的 template_file 提供程序來調用此腳本，我不得不轉義一些變量，因此出現了奇怪的插值。 但是我已經調試了腳本並且所有變量似乎都被正確放置了。 問題出在 SAS 代的某個地方，因為我不斷得到這個：

+ curl -H 'x-ms-date:Fri, 13 Sep 2019 11:04:40 GMT' -H x-ms-version:2009-09-19 -H x-ms-blob-type:BlockBlob -H 'Authorization: SharedKey *masked*:vyD7pp7Rqu3JBuS5IkHW0GMS2L82BN9fNKbmDAjuEoQ=' -H 'Content-
Type: application/octet-stream' -f https://*masked*.blob.core.windows.net/*masked*/*masked* -o *masked*
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

任何想法我可能在這里做錯了什么？

Answer 1

問題在於您的 string_to_sign 不完整，因為變量 canonicalized_resource 不完整。

您應該使用完整的 URL 聲明 canonicalized_resource，如下所示：

canonicalized_resource="/${STORAGE_ACCOUNT}/${STORAGE_CONTAINER}/${FILENAME}"

Answer 2

我無法讓它工作，所以我最終使用 terraform 文件提供程序來獲取節點上的文件，而不是從 Azure Blob 存儲中提取它們。

Answer 3

這對我有用（ 基於另一篇文章）：

#!/bin/bash
storage_account=$AZ_STG_ACCOUNT
container_name=$AZ_STG_CONTAINER
access_key=$AZ_STG_ACCESS_KEY
blob_name=PSCR_230113_000000Fri.xml

blob_store_url="blob.core.windows.net"
authorization="SharedKey"

request_method="GET"
request_date=$(TZ=GMT date "+%a, %d %h %Y %H:%M:%S %Z")
storage_service_version="2015-02-21"

# HTTP Request headers
x_ms_date_h="x-ms-date:$request_date"
x_ms_version_h="x-ms-version:$storage_service_version"

# Build the signature string
canonicalized_headers="${x_ms_date_h}\n${x_ms_version_h}"
canonicalized_resource="/${storage_account}/${container_name}/${blob_name}"

string_to_sign="${request_method}\n\n\n\n\n\n\n\n\n\n\n\n${canonicalized_headers}\n${canonicalized_resource}"

# Decode the Base64 encoded access key, convert to Hex.
decoded_hex_key="$(printf $access_key | base64 -d -w0 | xxd -p -c256)"

# Create the HMAC signature for the Authorization header
signature=$(printf "$string_to_sign" | openssl dgst -sha256 -mac HMAC -macopt "hexkey:$decoded_hex_key" -binary |  base64 -w0)

authorization_header="Authorization: $authorization $storage_account:$signature"
# -v or --trace to enable tracing
curl -v \
  -H "$x_ms_date_h" \
  -H "$x_ms_version_h" \
  -H "$authorization_header" \
  "https://${storage_account}.${blob_store_url}/${container_name}/${blob_name}" -o ${blob_name}