Why is flask's jsonify method slow?

Question

I'm writing an API in flask that returns json. Each flask function is of the form

from flask import jsonify
@app.route('/getdata')
def get_data():
    data = load_data_as_dict()
    return jsonify(data)

If I return a large amount of data, a call to this function takes around 1.7 seconds. However, if I do this:

from flask import Response
@app.route('/getdata')
def get_data():
    data = load_data_as_dict()
    data_as_str = json.dumps(data)
    return Response(response=data_as_str, status=200, mimetype="application/json"

...the function completes in around .05 seconds.

Can anyone tell me why jsonify is so much slower? Is there anything wrong with returning a raw Flask response instead?

Answer 1

My guess is: it has a lot to do with indentation and making a pretty json dump. Here's the method definition (I stripped the comments to save space, full code can be found here ) :

def jsonify(*args, **kwargs):
    indent = None
    separators = (',', ':')

    if current_app.config['JSONIFY_PRETTYPRINT_REGULAR'] and not request.is_xhr:
        indent = 2
        separators = (', ', ': ')

    if args and kwargs:
        raise TypeError('jsonify() behavior undefined when passed both args and kwargs')
    elif len(args) == 1:  # single args are passed directly to dumps()
        data = args[0]
    else:
        data = args or kwargs

    return current_app.response_class(
        (dumps(data, indent=indent, separators=separators), '\n'),
        mimetype=current_app.config['JSONIFY_MIMETYPE']
    )

dumps wraps simplejson.dumps if the module is available, otherwise it uses json.dumps .

Answer 2

jsonify() just wraps json.dumps() . However, depending upon the config of your Flask app and the Flask version that you're using, it may pass indent=2 and separators=(', ', ': ') to json.dumps . (See the docs on pretty-printing at https://docs.python.org/3/library/json.html if you're unfamiliar with these arguments).

Passing these arguments slows down json.dumps dramatically. Using the 181MB citylots.json file from https://github.com/zemirco/sf-city-lots-json as sample data, these pretty-printing arguments increase json.dumps() 's runtime from 7 seconds to 31 seconds on my MacBook Pro:

>>> import time 
>>> import json
>>> citylots = json.load(open('citylots.json'))
>>> start = time.time(); x = json.dumps(citylots); print(time.time() - start)
7.165302753448486
>>> x = None
>>> start = time.time(); x = json.dumps(citylots, indent=2, separators=(', ', ': ')); print(time.time() - start)
31.19125771522522

As of Flask 1.0, this costly pretty-printing will happen if either :

• You've explicitly set JSONIFY_PRETTYPRINT_REGULAR to True in your app's config (it's False by default), OR
• You're running your app in debug mode

(You can see these conditions in the Flask 1.0.2 code at https://github.com/pallets/flask/blob/1.0.2/flask/json/__init__.py#L309 .)

If you are using Flask >=1.0 and have the (probably unusual) need to disable the pretty-printing even in debug mode, you can always implement your own jsonify by copying and pasting the built-in jsonify 's definition and deleting all the pretty-printing logic:

from flask import current_app
from json import dumps

def jsonify(*args, **kwargs):
    if args and kwargs:
        raise TypeError('jsonify() behavior undefined when passed both args and kwargs')
    elif len(args) == 1:  # single args are passed directly to dumps()
        data = args[0]
    else:
        data = args or kwargs

    return current_app.response_class(
        dumps(data) + '\n',
        mimetype=current_app.config['JSONIFY_MIMETYPE']
    )

If you're in a version Flask prior to 1.0, then pretty-printing instead happens if both:

• You haven't explicitly set JSONIFY_PRETTYPRINT_REGULAR to False in you app's config (it's True by default), AND
• The current request is not an XHR request

In those older versions, there is never any need to redefine jsonify to eliminate the pretty-printing, since you can just do:

app.config['JSONIFY_PRETTYPRINT_REGULAR'] = False

(Alternatively, if you're using a pre-1.0 version of Flask and only want to disable the pretty-printing in production, then there's no need to change your code; instead, just upgrade to the latest version of Flask.)

Answer 3

It took me a while to figure out, but Flask jsonify sets the sort_keys argument on the encoder and it seems it defaults to True .

Adding:

JSON_SORT_KEYS = False

To the configuration gave me a factor 7 speed up for larger JSON structures.