Python Requests Returning 401 code on 'get' method

Question

I'm working on a webscrape function that's going to be pulling HTML data from internal (non public) servers. I have a connection through a VPN and proxy server so when I ping any public site I get code 200 no problem, but our internals are returning 401.

Heres my code:

http_str = f'http://username:password@proxy.yourorg.com:80'

proxyDict = {
    'http' : http_str, 
    'https' : https_str, 
    'ftp' : https_str
    }

html_text = requests.get(url, verify=True, proxies=proxyDict, auth=HTTPBasicAuth(user, pwd))

I've tried flushing my DNS server, using different certificate chains (that had a whole new list of problems). I'm using urllib3 on version 1.23 because that seemed to help with SSL errors. I've considered using a requests session but I'm not sure what that would change.

Also, the url's we're trying to access DO NOT require a log in. I'm not sure why its throwing 401 errors but the auth is for the proxy server, I think. Any help or idea are appreciated, along with questions as at this point I'm not even sure what to ask to move this along.

Edit: the proxyDict has a string with the user and pwd passed it for each type, https http fts, etc.

Answer 1

To use HTTP Basic Auth with your proxy, use the http://user:password@host/ syntax in any of the proxy configuration entries. See apidocs .

import requests
proxyDict = {
    "http": "http://username:password@proxy.yourorg.com:80",
    "https": "http://username:password@proxy.yourorg.com:80"
}
url = 'http://myorg.com/example'
response = requests.get(url, proxies=proxyDict)