python flask response using jsonify

Question

Two applications built with flask API, trying to receive a response with huge JSON response fails with Error 10054, 'An existing connection was forcibly closed by the remote host'

I could narrow the issue that when response is huge it fails

@api.route('/endpoint', methods=['POST'])
def endpoint():

   result = {small / huge dict}

   return jsonify({'result': result}), 200

caller side:

result = requests.post(url, params=data['args'], json=data['payload'])
        return result.json()['result']

Error log:

File "C:\Program Files (x86)\Python36-32\lib\http\client.py", line 1331, in getresponse
response.begin()
File "C:\Program Files (x86)\Python36-32\lib\http\client.py", line 321, in begin
self.headers = self.msg = parse_headers(self.fp)
File "C:\Program Files (x86)\Python36-32\lib\http\client.py", line 206, in parse_headers
line = fp.readline(_MAXLINE + 1)
File "C:\Program Files (x86)\Python36-32\lib\socket.py", line 586, in readinto
return self._sock.recv_into(b)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

Update:

tried to yield response as plan text and failures where much less, but still once in a while the issue appears.

def response(output):
    return Response(response_generator(output), mimetype='text/plain')


def response_generator(result):
    result_str = json.dumps(result)
    for row in [result_str[i:i + 1024*1024] for i in range(0, len(result_str), 1024*1024)]:
         yield row

Answer 1

As the Error log, the problems maybe one of below:

1. You are trying to open a url twice in your code. I don't see the whole caller side, so i cannot say if it's true or provide any further solution for sure

2. The request return bytes, which need to be decoded or Json decoding fail. I suggest you should change the code as below:

   result = requests.post(url, params=data['args'], json=data['payload']).text return result

If you need work with json, you should use:

result = json.loads(requests.post(url, params=data['args'], json=data['payload']).text)