如何在 fiddler 中捕获 python https 流量？

Question

Python throws in errors when ever I try to do some data fetching task. This only happens when I set fiddler to decrypt https traffic. I have tried routing python traffic through 127.0.0.1:8888 and same with mozilla inorder to catch its traffic. I also installed the certificate and trusted it via fiddler, I am not sure where I am going wrong.

    raise SSLError(e, request=request)
    requests.exceptions.SSLError: HTTPSConnectionPool(host='google.com', port=443):
    Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFIC
    ATE_VERIFY_FAILED] certificate verify failed (_ssl.c:748)'),))

This above is the error I get whenever I try to fetch a page with requests.

Answer 1

TL;DR The requests library does not use the windows certificate store, it has it's own one (as per https://bugs.python.org/issue28547 ). This means that your fiddler MITM certificate is not available to python requests by default.

Your options are

1. Disable SSL verification (verify=False)
2. Add your cert via the REQUESTS_CA_BUNDLE environment variable
3. Add your fiddler cert explicitly (verify='\\path\\to\\cert')

More details can be found at http://docs.python-requests.org/en/master/user/advanced/#ssl-cert-verification

On a side note, it does feel a little strange for requests to be using it's own cert bundle, rather than the platform supplied one - especially given all the browsers are happy to use the platform ones.

Answer 2

As pointed out by polhemic and Eric Aronesty, for testing purposes, you can set temporarily "CURL_CA_BUNDLE" to an empty string.

import os
os.environ['CURL_CA_BUNDLE'] = ''