遍历python代理字典

Question

So I am using the requests module, but I am trying to change the proxy every time a request is made (Ex, GET and POST). I have a dictionary of all the proxies I want to use, but I am having trouble getting the request to actually work through iterating through the dictionary. I understand how to send a request with a single proxy, but again, I am not sure how to with changing each proxy after every request. This is not the current program I am trying to write, but similarly the task I am trying to accomplish:

BASE_URL = "Some url"
USER_AGENT = "Some user agent"
POST_URL = "Some url"

proxies = {
    'https' : 'proxy1',
    'https' : 'proxy2',
    'https' : 'proxy...'
}


def req():

        session = requests.Session()
        session.headers = {'user-agent': USER_AGENT}
        session.headers.update({'Referer': BASE_URL})
        req = session.get(BASE_URL, proxies=curProxy)

        session.headers.update({'x-csrftoken': req.cookies['csrftoken']})
        login_data = {'DATA HERE'}
        login = session.post(POST_URL, data=login_data, allow_redirects=True, proxies=curProxy)
        session.headers.update({'x-csrftoken': login.cookies['csrftoken']})
        cookies = login.cookies



# For each proxy in proxies
for proxy in proxies:
    # Updating the proxy to use
    curProxy = proxy
    req()

Thanks to all who reply in advance. All help/input is greatly appreciated!

Answer 1

You don't need a dictionary for your proxies. Use a plain list:

proxies = ['proxy1', 'proxy2', ...]

Change your function req to accept the proxy as a parameter. Global variables are evil :)

def req(curProxy):
    ...
    req = session.get(BASE_URL, proxies={'http': curProxy, 'https': curProxy})

Then iterate

for proxy in proxies:
    req(proxy)