Can´t download azure blob using proxies from python

Question

I´m trying to download a azure blob from a server which uses proxy. Right now, my code works perfectly in my local (without using proxy as it is not needed) but when I try to run it from a server where I need to use the proxy, it doesn´t work even though I followed the azure documentation. I get the following error:

azure.core.exceptions.ClientAuthenticationError: Authentication failed: <urllib3.connection.HTTPSConnection object at 0x7fa5b831f760>: Failed to establish a new connection: [Errno -2] Name or service not known

This is the code:

from azure.identity import ClientSecretCredential
from azure.storage.blob import BlobServiceClient 
credential = ClientSecretCredential(self.tenant_id, self.client_id, self.client_secret)
proxy = {"http": proxy_value, "https": proxy_value}
blob_service_client_instance = BlobServiceClient(
    account_url=storage_account_url, credential=credential, proxies=proxy, connection_verify=False
)
blob_client_instance = blob_service_client_instance.get_blob_client(
    container_name, blob_name, snapshot=None
)
blob_data = blob_client_instance.download_blob()

I´m 100% sure that all the values are correct as I have done other different tests that work.

Thanks in Advance!!

Answer 1

I tried in my environment and got below results:

Initially I tried in my environment and got error,

(Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x04755510>: Failed to establish a new connection: [Errno11001] getaddrinfo failed',)))

The error occurs due to your proxy password may contains an @ symbol , which is causing your URL parsing to get confused. You need to URL encode your password (that is, run it through urllib.quote) before passing it or it gets pretty confusing.

You can refer this document to set variable in the azure cli. I tried with urllib.parse with below python code I downloaded and read the content of file with proxy successfully.

from azure.identity import ClientSecretCredential
from azure.storage.blob import BlobServiceClient
import json
import os
import requests
from urllib.parse import quote


username = quote('type in username')  
password = quote(' type in password')  
proxy = 'type in proxy name or ip'  
proxy_port = '8080'

proxy_https = 'https://' + username + ':' + password + '@' + proxy + ':' + proxy_port

proxies={ 'https': proxy_https}
tenant_id="<tenant id>"
client_id="<client id>"
client_secret="<client secret>"

credential = ClientSecretCredential(tenant_id,client_id,client_secret)
storage_url="https://venkat12.blob.core.windows.net"

container_name="test"
blob_name="file.json"
blob_service_client_instance = BlobServiceClient(account_url=storage_url,proxy=proxies,credential=credential,connection_verify=True)
blob_client_instance = blob_service_client_instance.get_blob_client(
    container_name,blob_name, snapshot=None
)
blob_data = blob_client_instance.download_blob().readall()
raw=json.loads(blob_data)
print(raw)

Console:

Reference: Can't get Proxy to work with https and http works only with auth, not like the Docu descripes · Issue #3990 · psf/requests (github.com) by eisbaer9.