How to send POST request with no data

Question

Is it possible using urllib or urllib2 to send no data with a POST request? Sounds odd, but the API I am using sends blank data in the POST request.

I've tried the following, but it seems to be issuing a GET request because of no POST data.

url = 'https://site.com/registerclaim?cid=' + int(cid)
values = {}
headers = {
    'User-Agent' : 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36',
    'X-CRFS-Token' : csrfToken,
    'X-Requested-With' : 'XMLHttpRequest'   
}

data    = urllib.urlencode(values)
req     = urllib2.Request(url, data, headers)
resp    = opener.open(req)

I am getting a 404 exception, which is what the API returns if trying to access the page via a 'GET' request. I've checked all the variables to ensure they are set correctly, and they are.

Am advice?

Answer 1

Your diagnosis is incorrect; urllib2 will send an empty POST body in your case:

>>> import urllib, urllib2
>>> url = 'http://httpbin.org/post?cid=42'
>>> headers = {
...     'User-Agent' : 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36',
...     'X-CRFS-Token': 'csrf_token_mocked',
...     'X-Requested-With' : 'XMLHttpRequest'   
... }
>>> values = {}
>>> data    = urllib.urlencode(values)
>>> req     = urllib2.Request(url, data, headers)
>>> req.has_data()
True
>>> req.get_method()
'POST'
>>> resp    = urllib2.urlopen(req)
>>> body = resp.read()
>>> print body
{"args": {"cid": "42"}, "data": "", "files": {}, "form": {}, "headers": {"Accept-Encoding": "identity", "Connection": "close", "Content-Length": "0", "Content-Type": "application/x-www-form-urlencoded", "Host": "httpbin.org", "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36", "X-Crfs-Token": "csrf_token_mocked", "X-Request-Id": "a14f84f5-a355-4b8a-8b34-cb42808b8b09", "X-Requested-With": "XMLHttpRequest"}, "json": null, "origin": "81.134.152.4", "url": "http://httpbin.org/post?cid=42"}
>>> from pprint import pprint
>>> import json
>>> pprint(json.loads(body))
{u'args': {u'cid': u'42'},
 u'data': u'',
 u'files': {},
 u'form': {},
 u'headers': {u'Accept-Encoding': u'identity',
              u'Connection': u'close',
              u'Content-Length': u'0',
              u'Content-Type': u'application/x-www-form-urlencoded',
              u'Host': u'httpbin.org',
              u'User-Agent': u'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36',
              u'X-Crfs-Token': u'csrf_token_mocked',
              u'X-Request-Id': u'a14f84f5-a355-4b8a-8b34-cb42808b8b09',
              u'X-Requested-With': u'XMLHttpRequest'},
 u'json': None,
 u'origin': u'81.134.152.4',
 u'url': u'http://httpbin.org/post?cid=42'}

Some things to note:

• the http://httpbin.org/post route only accepts POST methods; it'll return a 405 response otherwise.
• The req.get_method() method is used to determine what method to use when sending the request; you can see that it'll use POST even though values is empty. req.get_method() determines the method used based on the req.has_data() response, and that method returns True when the data attribute is not None . An empty string qualifies here as having data.
• The http://httpbin.org/post response shows the request headers; the content-length header was set to 0, indicating an empty POST body.

So the question then is: how certain are you you have everything required for the POST to succeed? Perhaps a Referer header is required, or perhaps you misunderstood what parameters are passed in and the POST body is not meant to be empty.