Python Requests Not Encoding Forward Slashes

Question

I'm working on a tool that needs to make requests to a site using the application/x-www-form-urlencoded content type. When I make the request from ARC for Chrome , I get the expected response. When I reproduce the request using requests for Python 3, I get a different response that is not at all what I expected.

I opened WireShark to find differences between the request coming from ARC and the request coming from the Python script. The content of the URL-encoded form, from WireShark, for the ARC request is as follows:

TransType=INQ&TransID=RESINQ&ReturnPage=%2Fdmvnet%2Fplate_purchase%2Fs2end.asp&HelpPage=&Choice=A&PltNo=TSPOON&HoldISA=N&HoldSavePltNo=&HoldCallHost=&NumCharsInt=8&CurrentTrans=plate_purchase_reserve&PltType=IGWT&PersonalMsg=Y&Let1=T&Let2=S&Let3=P&Let4=O&Let5=O&Let6=N&Let7=&Let8=

The content of the URL-encoded form, from WireShark, for the Python script request is as follows:

TransType=INQ&TransID=RESINQ&ReturnPage=/dmvnet/plate_purchase/s2end.asp&HelpPage=&Choice=A&PltNo=TSPOON&HoldISA=N&HoldSavePltNo=&HoldCallHost=&NumCharsInt=8&CurrentTrans=plate_purchase_reserve&PltType=IGWT&PersonalMsg=Y&Let1=T&Let2=S&Let3=P&Let4=O&Let5=O&Let6=N&Let7=&Let8=

The only difference that I see (and that a diff checker reports) between these two is the ReturnPage . In the request from ARC, the forward slashes are translated to %2F , and in the request from the Python script they remain unencoded. Shouldn't the requests library be performing this encoding? When I use a raw string in Python the %2F itself gets encoded ( ReturnPage=%252Fdmvnet%252Fplate_purchase%252Fs2end.asp ). Does the lack of encoding from the requests library even matter here?

I am using the application/x-www-form-urlencoded header...

headers = {
    "content-type": "application/x-www-form-urlencoded"
}

body = {
    ...
}

response = requests.post("myUrlHere", data=body, headers=headers)

The response that comes back from the Python script using requests is as follows:

Status Code: 200
Headers: {
    'Cache-Control': 'private',
    'Content-Type': 'text/html',
    'Server': '',
    'Set-Cookie': 'WebSessionDataID=20200816233709078125172161292511; path=/, ASPSESSIONIDQERAATBR=ODECNAOABGCGADAJEMCCNIDM; secure; path=/',
    'Date': 'Mon, 17 Aug 2020 03: 37: 08 GMT', 'Content-Length': '0'
},
Content: "" (Empty)

The anticipated response is a 302 Object Moved with a destination of the requested resource provided in the response headers.

Answer 1

It has nothing to do with the encoding of forward slashes. According to the documentation :

By default Requests will perform location redirection for all verbs except HEAD.

This means that if a 302 (Found) is received (or, in general, any 3xx response status code), the request is automatically redirected to the received location. If you do not want this, the same documentation says:

If you're using GET, OPTIONS, POST, PUT, PATCH or DELETE, you can disable redirection handling with the allow_redirects parameter:

And provides this example:

>>> r = requests.get('http://github.com/', allow_redirects=False)

>>> r.status_code
301

>>> r.history
[]