通过AWS服务器上的代理的HTTP请求失败并显示407

Question

I'm running this brief script locally as well on an AWS EC2 intance in an ECS cluster:

 import requests 
 proxies = {'http':'http://user:pw@host:port','https':'http://user:pw@host:port'}
 r = requests.get('http://quotes.toscrape.com/', proxies=proxies)
 print r.status_code

When I run the script locally, I get a 200 status code, indicating that I successfully am able to connect to access the website via the proxy.

When I run the script on the AWS instance, I get a 407 proxy authentication error.

This is a common error that others have experienced (eg see here ). I'm looking for a solution that allows me to authenticate the proxies WITHOUT having to whitelist the instance.

The reason being is that every time I run a new instance, I'd have to whitelist that instance too. I would rather just pass in the credentials to requests and authenticate the proxy directly.

Is this possible?

Answer 1

I would suggest you launch instances in the private subnet, and whitelist your NAT(s) EIP in this case you will only have to whitelist 1-4 IP addresses depending if you are using single NAT or a NAT per AZ (which is recommended).

Hopefully, it does make sense, feel free to ask additional questions