Python Requests XML API Exception ConnectionError: ('Connection aborted.', BadStatusLine(“''”,)) when xml size increases

Question

I am trying to hit an XML API. It works fine when I am using small XML payload.

Below is the code

import requests

xml = """<?xml version="1.0" encoding="ISO-8859-1"?>
     <!DOCTYPE MESSAGE SYSTEM "http://127.0.0.1:80/psms/dtd/messagev12.dtd">
     <MESSAGE VER="1.2">
     <USER USERNAME="xxxxx" PASSWORD="xxxx" />
     <CONTENT  UDH="0" CODING="1" TEXT="SAMPLE" PROPERTY="0" ID="1" TEMPLATE="">
     <ADDRESS FROM="XXXX" TO="XXXXXXXXXX" SEQ="0" TAG="test"/>
     ......
     ......
     </CONTENT>
     </MESSAGE>"""

     payload = {'data'   : xml,'action'   : 'send'}
     headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.0; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0',
               'Content-Type': 'application/xml'}
     response =  requests.post(url, params = payload, headers=headers)
     print response.status_code
     print response.content

The moment XML gets bigger with more data following Exception is raised

Traceback (most recent call last):
File "C:\Users\dell\Desktop\SMS XML\xml_sms_api.py", line 30, in <module>
response =  requests.post(url, params = payload, headers=headers)
File "C:\Python27\lib\site-packages\requests\api.py", line 110, in post
return request('post', url, data=data, json=json, **kwargs)
File "C:\Python27\lib\site-packages\requests\api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "C:\Python27\lib\site-packages\requests\adapters.py", line 473, in send
raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', BadStatusLine("''",))

It works absolutely fine till payload XML size is small. But the API is verified work up to 5k entries in single XML.

Answer 1

After trying out various options found the problem and solution.

Since XML is provided as params in Requests it is getting URL encoded into the URL. So when huge XML data is passed URL length crosses beyond limit.

So changed the params into data also the content type

payload = {'data'   : xml,'action'   : 'send'}
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.0; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0',
           'Content-Type': 'application/application/x-www-form-urlencoded'}
response =  requests.post(url, data = payload, headers=headers)