What AWS service can help speed up slow Django view functions

Question

I have a Django application with a Postgres DB that has kind of run away from my client as the database has millions of records. That being said there are some view functions that make several database queries, and because of that, it is glacier slow. Here is an example of one of my view functions It is kind of messy because I was playing around with things to look for query optimizations, my apologies.

def get_filters(request):
    try:
        post_data = request.body.decode("utf-8")
        post_data = json.loads(post_data)
    except Exception as e:
        logging.error('There was an issue getting filter post data')
        logging.error(e)

        return HttpResponse(
            json.dumps({'filters': {'makes': [], 'models': [], 'years': [], 'transmissions': [], 'fuel_types': []}}),
            content_type='application/json')

    dealer = post_data['dealer'] if 'dealer' in post_data else None
    make_ids = post_data['make_ids'] if 'make_ids' in post_data else []
    model_ids = post_data['model_ids'] if 'model_ids' in post_data else []
    years = post_data['years'] if 'years' in post_data else []
    condition = post_data['condition'] if 'condition' in post_data else None
    price_min = post_data['price_min'] if 'price_min' in post_data else 0
    price_max = post_data['price_max'] if 'price_max' in post_data else 10000000

    # Catch Critical Error Where There Is No Dealer
    if not dealer:
        logging.error('Unable to find a Dealer')
        return HttpResponse(
            json.dumps({'filters': {'makes': [], 'models': [], 'years': [], 'transmissions': [], 'fuel_types': []}}),
            content_type='application/json')
    else:
        try:
            dealer = Dealer.objects.get(id=dealer)
        except Exception as e:
            logging.error(e)
            return HttpResponse(
                json.dumps({'filters': {'makes': [], 'models': [], 'years': [], 'transmissions': [], 'fuel_types': []}}),
                content_type='application/json')


    current_year = datetime.datetime.now().year
    start_year = current_year - 30

    # First get the make filters
    vehicles = Vehicle.objects.filter(dealer=dealer)

    if years:
        vehicles = vehicles.filter(year__in=years)

    filtered_make_names = vehicles.values_list('vehicle_make__name', flat=True)
    filtered_makes = VehicleMake.objects.filter(name__in=filtered_make_names)

    makes_map = [{
            'name': make.name,
            'count': vehicles.filter(vehicle_make=make, dealer=dealer).count(),
            'id': make.id
        } for make in filtered_makes
    ]

    # Second get the model filters

    filtered_model_names = vehicles.values_list('vehicle_model__name', flat=True)
    filtered_models = VehicleMake.objects.filter(name__in=filtered_model_names)

    dealer_models = VehicleModel.objects.filter(
        name__in=filtered_models)

    new_dealer_models = VehicleModel.objects.filter(name__in=vehicles.values_list('vehicle_model__name', flat=True))


    if len(make_ids) > 0:
        dealer_models = dealer_models.filter(make__id__in=make_ids)

    # Get the actual filters
    year_map = [{
        'year': yr,
        'count': Vehicle.objects.filter(year=yr, dealer=dealer).count()
    } for yr in range(start_year, current_year) if Vehicle.objects.filter(year=yr, dealer=dealer).count() > 0][::-1]

    models_map = [{
        'name': model.name,
        'count': Vehicle.objects.filter(vehicle_model=model, dealer=dealer).count(),
        'id': model.id
    } for model in dealer_models]

    filter_map = {
        "makes": makes_map,
        "models": models_map,
        "years": year_map,
        "transmissions": [],
        "fuel_types": []
    }

    return HttpResponse(json.dumps({'filters': filter_map}), content_type='application/json')

I want to launch and AWS EC2 instance, and migrate my code from the server I have now to there, and was wondering what AWS services I could use in conjunction with that to make those view functions faster, and why ? Does autoscaling assist with that, or does autoscaling only kick in when the CPU has hit a certain point ?

Answer 1

You need to figure out if the bottleneck is in your database layer, or in your web layer.

If the bottleneck is you database layer, than a bigger db instance on its own server or the introduction of a caching layer such as memcache or redis might be appropriate (django has plugs for both of these)

If the bottleneck is your website, than a combination of a load-balancer, multiple ec2 instances running you website, and an autoscaling group might be appropriate.

But first, you really need to figure out where the bottleneck is so you don't spend time and money optimizing the wrong thing.