Django Rest Framework订购Serializer.data

Question

I have a 3 table and they releated between each other. Also i have some serializers.SerializerMethodField which is "elapsed_time" this field calculating elapsed_time so how can i order my Entites according to that MethodField. One way that i found i'm sorting serializer.data this returns OrderedList but this way taking so much time for 2k record.

serializer = self.get_serializer(entites, many=True)
result_page = get_pool_ordered_queryset(serializer.data)

get_pool_ordered_queryset function takes OrderedDict and sorting.

serializer_data = sorted(serializer_data, key=lambda k: k['elapsed_time'], reverse=True)

Like this but when i add 2k record its taking almost 2 minutes.

Answer 1

I guess your question is how to speed up the process.

First of all you don't pass OrderedDict to get_pool_ordered_queryset , you pass list of dict values, that's what serializer.data returns.

Then you spent most of your time, computing 2k times your elapsed_time . You should try to move this part of computation to SQL and sort by it there.

You should probably be using annotate of django queryset, and the order by it.

This would drastically decrease your computation time.

Answer 2

The sorted() function may not be the time consumption factor here. You said there are 3 related tables. Django Queryset's are Lazy , so the 2k items from 3 related tables might consume around 2k x 3 DB connections, which is highly expensive/ time-consuming

Example :

In [1]: def get_time(mylist):
   ...:     pre_time = time.time()
   ...:     sorted(mylist, key=lambda x: x['value'])
   ...:     return "time consumed {}".format(time.time() - pre_time)
   ...: 
   ...: 

In [2]: 
   ...: import time
   ...: from random import randint
   ...: 
   ...: 

In [3]: 
   ...: mylist = []  # creates a list of dict as [ {"value":3},{"value":1},{"value":32},...]
   ...: for i in range(4000):
   ...:     mylist.append({"value": randint(1, 999999)})
   ...:     

In [4]: get_time(mylist)
Out[4]: 'time consumed 0.007257938385009766'

From this example we can understand, The time consumption is purely because of Django Queryset Lazyness .

In order to speed up your queries, you need some kind of optimization. I would like to suggest reading the optimization techniques that are specified in Django Doc will help you