Curl post request returning 500 server error in python

Question

The following curl request works perfectly when I run it through Windows' CMD.

curl -XPOST -H"Content-Type:application/json" http://my_url:8082/druid/v2/sql/ -d "{\"query\":\"SELECT DISTINCT(event) FROM programs\"}"

I am trying to replicate the same call in python 3 using urllib.requests

import urllib.request

values = {'query':'SELECT DISTINCT(event) FROM programs'}
url = 'http://my_url:8082/druid/v2/sql'

data = urllib.parse.urlencode(values).encode("utf-8")
req = urllib.request.Request(url, data)
req.add_header("Content-Type","application/json")
response = urllib.request.urlopen(req)
the_page = response.read()

However the python version is returning a Server Error

response = urllib.request.urlopen(req)
File "C:\Python\Python35\lib\urllib\request.py", line 163, in urlopen
return opener.open(url, data, timeout)
File "C:\Python\Python35\lib\urllib\request.py", line 472, in open
response = meth(req, response)
File "C:\Python\Python35\lib\urllib\request.py", line 582, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Python\Python35\lib\urllib\request.py", line 510, in error
return self._call_chain(*args)
File "C:\Python\Python35\lib\urllib\request.py", line 444, in _call_chain
result = func(*args)
File "C:\Python\Python35\lib\urllib\request.py", line 590, in 
http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 500: Server Error

Can someone tell me what I'm doing wrong please?

Answer 1

You need to convert your data to json first, you are sending a dictionary.

import json

data = urllib.parse.urlencode(json.dumps(values)).encode("utf-8")

Or, use the requests library :

import requests

d = {'query':'SELECT DISTINCT(event) FROM programs'}
url = 'http://my_url:8082/druid/v2/sql'

r = requests.post(url, json=d)
r.raise_for_status()

print(r.text)

Answer 2

After some more experimentation, I managed to make the code work by replacing urllib.parse.urlencode with json.dumps. The code now reads as follows:

import urllib.request, json

values = {'query':'SELECT DISTINCT(event) FROM programs'}
url = 'http://my_url/druid/v2/sql'

data = json.dumps(values).encode("utf-8")
req = urllib.request.Request(url, data)
req.add_header("Content-Type","application/json")
response = urllib.request.urlopen(req)
the_page = response.read()