Python requests.post very slow

Question

I am currently using requests to push a file via API to my SaaS product.

    file = {'file': open(file_name1, 'rb')}
headers = {'Authorization': 'Bearer ' + app_key}
response = requests.post(url=push_chunk_url ,files=file, headers=headers)

However, the upload speed is very slow (~1mbs) despite 150mbs upload available to the browser.

This response from 6 years ago suggests that the block size of httplib may be responsible: https://stackoverflow.com/a/39518613

Since that was written Python 3.7 introduced a blocksize parameter to httplib replacement http.client https://docs.python.org/3/library/http.client.html

I can't work out though how to possible pass the blocksize parameter from requests to set this higher and hopefully solve this issue.

Answer 1

According to the Github Issue( Requests is slow to upload large files ) it's not possible to passed down the blocksize parameter down to the http.client.HTTPConnection . The workaround solution is to monkey patch the HTTPConnection.__init__.__defaults__ with your block size(here we are using 64*1024 ).

from http.client import HTTPConnection
HTTPConnection.__init__.__defaults__ = tuple(
    x if x != 8192 else 64*1024
    for x in HTTPConnection.__init__.__defaults__
)