How to make Python Requests follow a POST redirect?

Question

Using the following code, the server responds with a 301 redirect, but the client changes POST to GET, which is useless behavior because that GET endpoint does not exist. Using CURL -L -X POST works properly. This behavior is the same using python2 and python3 and on several versions of Raspbian.

>>> import requests
>>> url = "https://registry.micronets.in/mud/v1/register- 
device/DAWG/AgoNDQcDDgg/aabbccddeeffgg"
>>> response = requests.post(url)
>>> response
<Response [404]>

# Server Log: (Note - both endpoints, are on the same server using virtual hosts)
redirecting to: https://hotdawg.micronets.in/registry/devices/register- device/AgoNDQcDDgg/aabbccddeeffgg
POST /registry/v1/register-device/DAWG/AgoNDQcDDgg/aabbccddeeffgg 301 16.563 ms - 122
{
  "status": 404
}
GET /vendors//register-device/AgoNDQcDDgg/aabbccddeeffgg 404 0.604 ms - 14

# CURL version (succeeds)
curl -L -X POST "https://registry.micronets.in/mud/v1/register- 
device/DAWG/AgoNDQcDDgg/aabbccddeeffgg"
Device registered (insert): {
  "model": "AgoNDQcDDgg",
  "pubkey": "aabbccddeeffgg",
  "timestamp": "2019-12-27 15:44:14 UTC",
  "_id": "HBlQzXfBnoB3N4fN"
}

# Server Log: (from CURL)
redirecting to: https://hotdawg.micronets.in/registry/devices/register- 
device/AgoNDQcDDgg/aabbccddeeffgg
POST /registry/v1/register-device/DAWG/AgoNDQcDDgg/aabbccddeeffgg 301 0.364 ms - 122
POST /vendors//register-device/AgoNDQcDDgg/aabbccddeeffgg 200 1.745 ms - 157

Answer 1

I'd rather accept a better answer, but otherwise I plan to work around the problem as follows:

response = requests.post(url, allow_redirects=False)
if response.status_code == 301:
    response = requests.post(response.headers['Location'])

or

response = requests.post(url, allow_redirects=False)
i=10
while i > 0 and response.status_code == 301:
    response = requests.post(response.headers['Location'], allow_redirects=False)
    i -= 1

Answer 2

Check your nginx configuration: Reference

1. You've already sent a POST request which is already reached the server successfully.
2. Now the server is handling the POST request and handling it according to your assigned rules.
3. Here's the issue occurred as the redirection assignment on your configuration is calling the end url by GET request.
4. You are blaming requests that it's not handling the redirect but it's already handled it correctly. but you compare it to curl which you used with it -L which is force the server the server by default to push the request. (etc, curl --list-only -X POST "https://registry.micronets.in/mud/v1/register-device/DAWG/AgoNDQcDDgg/aabbccddeeffgg" ) where you are using -X which is for HTTP over an HTTPS url.

-X, --request will be used for all requests, which if you for example use -L, --location may cause unintended side-effects when curl doesn't change request method according to the HTTP 30x response codes - and similar. <<< Pay attention to the part of curl doesn't change request method which is POST in your case and already forced the server with it. but for requests it's completely different thing.

import requests

with requests.Session() as ses:
    r = ses.post(
        "https://registry.micronets.in/mud/v1/register-device/DAWG/AgoNDQcDDgg/aabbccddeeffgg", allow_redirects=True)
    print(r.history[0].headers['Location'])

Output:

https://hotdawg.micronets.in/registry/devices/register-device/AgoNDQcDDgg/aabbccddeeffgg

at the end, i do believe this nginx server which is behind linode servers, and that's a common issue.

the knowledge is here:

import requests

with requests.Session() as ses:
    r = ses.post(
        "https://registry.micronets.in/mud/v1/register-device/DAWG/AgoNDQcDDgg/aabbccddeeffgg", allow_redirects=True)
    print(r.history, r.history[0].reason)
    print(r.status_code, r.reason)

Output:

[<Response [301]>] Moved Permanently
404 Not Found

But POST To it !

r = requests.post(
    "https://hotdawg.micronets.in/registry/devices/register-device/AgoNDQcDDgg/aabbccddeeffgg")
print(r)

Output:

<Response [200]>

Which confirm that you sent POST request and it's completely switched to GET .

with that said, as i explained above that curl -X is set to force the server with the method used which is POST till the end point.