Developing REST APIs using Python (Sanic)

Question

from sanic import Sanic
from sanic import response


app = Sanic(__name__)

@app.route('/v1/ok', methods=['GET'])
async def post_handler(request):
    return response.text("hey all good")

if __name__ == '__main__':
        app.run(host="0.0.0.0", port=8001, debug=True)

I am trying to write REST APIs in python using sanic

Here are my conclusions:

I tried benchmarking this GET API using wrk with 50 threads running 30s test. Machine used AWS EC2 t2.medium which has 4GB RAM and 2 CPU command used

wrk -t50 -c4000 -d30s http://XXX.XX.XXX.XXX:8001/v1/ok

benchmarking result

Running 30s test @ http://XXX.XX.XXX.XXX:8001/v1/ok
 50 threads and 4000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   559.30ms  117.86ms   1.99s    94.47%
    Req/Sec    41.92     44.33   361.00     86.14%
  53260 requests in 30.10s, 6.70MB read
  Socket errors: connect 1493, read 15631, write 0, timeout 4
Requests/sec:   1769.21
Transfer/sec:    228.06KB

My doubt is that, how could i improve on

1. the number of requests that gets timed out. Currently it is 4. It should be zero.
2. average latency, which is ~550ms (too much)

In the case of POST request, it is very very bad, wherein I am trying to load a keras model and do the predictions.

Is it a problem in a way the code is written?

OR

Is this a limitation of Sanic?

Should I try another REST framework?

PS: My experience with flask is even worse in terms of latency and timed out requests.

import sys
import os
import json
import pandas
import numpy
import optparse
from keras.models import Sequential, load_model
from keras.preprocessing import sequence
from keras.preprocessing.text import Tokenizer
from collections import OrderedDict
from sanic import Sanic
from sanic import response
import time

app = Sanic(__name__)

@app.route('/v1/mal/prediction', methods=['POST'])
async def post_handler(request):
    csv_file = 'alerts.csv'
    log_entry = request.json
    dataframe = pandas.read_csv(csv_file, engine='python', quotechar='|', header=None)
    dataset = dataframe.values
    X = dataset[:,0]
    for index, item in enumerate(X):
        reqJson = json.loads(item, object_pairs_hook=OrderedDict)
        del reqJson['timestamp']
        del reqJson['headers']
        del reqJson['source']
        del reqJson['route']
        del reqJson['responsePayload']
        X[index] = json.dumps(reqJson, separators=(',', ':'))

    tokenizer = Tokenizer(filters='\t\n', char_level=True)
    tokenizer.fit_on_texts(X)
    seq = tokenizer.texts_to_sequences([log_entry])
    max_log_length = 1024
    log_entry_processed = sequence.pad_sequences(seq, maxlen=max_log_length)
    model = load_model('model.h5')
    model.load_weights('weights.h5')
    model.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics = ['accuracy'])
    prediction = model.predict(log_entry_processed)
    return response.text(prediction[0])

if __name__ == '__main__':
        app.run(host="0.0.0.0", port=8000, debug=True)

Please suggest better ways to improve the API response time and decreasing the timed out requests?

Answer 1

禁用debug并将workers设置为您实例中的CPU数量（t2.med为2）：

app.run(host="0.0.0.0", port=8001, workers=2)

Answer 2

A little late to the game here, but I believe in order to make it truly asynchronous you need to add await calls. Otherwise you're just calling blocking functions.