Logging requests to django-rest-framework

Question

For debugging purposes, I would like to use Django's logging mechanism to log each and every incoming request when it "arrives" at django-rest-framework's doorstep.

Djagno offers logging of its requests (only "warning" log level and above) in the following manner (from LOGGING section in settings.py):

'django.request': {
        'handlers': ['mail_admins'],
        'level': 'ERROR',
        'propagate': False,
 },

I'm looking to achieve something like this (notice: log level is DEBUG):

'rest_framework.request': {
        'handlers': ['logfile'],
        'level': 'DEBUG',
        'propagate': False,
 },

Is there a way I can do that without embedding a logger in to DRF's source code?
Is there maybe some sort of "Logging Backend" option in DRF I'm not aware of?

Answer 1

I made a generic RequestLogMiddleware that can be hooked into any Django View using decorator_from_middleware .

request_log/middleware.py

import socket
import time


class RequestLogMiddleware(object):
    def process_request(self, request):
        request.start_time = time.time()

    def process_response(self, request, response):

        if response['content-type'] == 'application/json':
            if getattr(response, 'streaming', False):
                response_body = '<<<Streaming>>>'
            else:
                response_body = response.content
        else:
            response_body = '<<<Not JSON>>>'

        log_data = {
            'user': request.user.pk,

            'remote_address': request.META['REMOTE_ADDR'],
            'server_hostname': socket.gethostname(),

            'request_method': request.method,
            'request_path': request.get_full_path(),
            'request_body': request.body,

            'response_status': response.status_code,
            'response_body': response_body,

            'run_time': time.time() - request.start_time,
        }

        # save log_data in some way

        return response

request_log/mixins.py

from django.utils.decorators import decorator_from_middleware

from .middleware import RequestLogMiddleware


class RequestLogViewMixin(object):
    """
    Adds RequestLogMiddleware to any Django View by overriding as_view.
    """

    @classmethod
    def as_view(cls, *args, **kwargs):
        view = super(RequestLogViewMixin, cls).as_view(*args, **kwargs)
        view = decorator_from_middleware(RequestLogMiddleware)(view)
        return view

my_django_rest_api/views.py

from rest_framework import generics

from ...request_log.mixins import RequestLogViewMixin

class SomeListView(
    RequestLogViewMixin,
    generics.ListAPIView
):
    ...

Answer 2

Override the APIView.initial() method to add logging yourself.

Dispatch methods

The following methods are called directly by the view's .dispatch() method. These perform any actions that need to occur before or after calling the handler methods such as .get(), .post(), put(), patch() and .delete().

.initial(self, request, *args, **kwargs)
Performs any actions that need to occur before the handler method gets called. This method is used to enforce permissions and throttling, and perform content negotiation.

Answer 3

Here my current solution to get every REQUEST/RESPONSE in the log. I created a middleware compatible with the old middleware (Django < 1.10) and the new middleware that log every request/response. This solution is the best I found so far.

import logging
from django.utils.deprecation import MiddlewareMixin

_logger = logging.getLogger(__name__)

class LogRestMiddleware(MiddlewareMixin):
    """Middleware to log every request/response.
    Is not triggered when the request/response is managed using the cache
    """

    def _log_request(self, request):
        """Log the request"""
        user = str(getattr(request, 'user', ''))
        method = str(getattr(request, 'method', '')).upper()
        request_path = str(getattr(request, 'path', ''))
        query_params = str(["%s: %s" %(k,v) for k, v in request.GET.items()])
        query_params = query_params if query_params else ''

        _logger.debug("req: (%s) [%s] %s %s", user, method, request_path, query_params)

    def _log_response(self, request, response):
        """Log the response using values from the request"""
        user = str(getattr(request, 'user', ''))
        method = str(getattr(request, 'method', '')).upper()
        status_code = str(getattr(response, 'status_code', ''))
        status_text = str(getattr(response, 'status_text', ''))
        request_path = str(getattr(request, 'path', ''))
        size = str(len(response.content))

        _logger.debug("res: (%s) [%s] %s - %s (%s / %s)", user, method, request_path, status_code, status_text, size)

    def process_response(self, request, response):
        """Method call when the middleware is used in the `MIDDLEWARE_CLASSES` option in the settings. Django < 1.10"""
        self._log_request(request)
        self._log_response(request, response)
        return response

    def __call__(self, request):
        """Method call when the middleware is used in the `MIDDLEWARE` option in the settings (Django >= 1.10)"""
        self._log_request(request)
        response = self.get_response(request)
        self._log_response(request, response)
        return response

Answer 4

Here is the code from @Glyn Jackson's Answer:

in middleware/mixin.py

class RequestLogMiddleware(object):

    def initial(self, request, *args, **kwargs):
         super(RequestLogMiddleware, self).initial(request, *args, **kwargs)
         # store/log the request

in the View:

class ViewClass(RequestLogMiddleware, generics.RetrieveAPIView):
     ...

Answer 5

In new Django 2+ better to use Middleware as a callable object like this, just connect it with your project in your Middlewares section of the settings.py file (also Middleware can be a function, not only a class because it's a callable object old MiddlewareMixin now in the deprecated module of Django):

More info in docs:
https://docs.djangoproject.com/en/2.2/topics/http/middleware/#writing-your-own-middleware

class UserActivityLogMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        print(request.method)    # In this string we catch request object.
        response = self.get_response(request)
        return response

Answer 6

Made a custom middleware:

import logging
import time
logger = logging.getLogger(__name__)

class APILogMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response
    
    def __call__(self, request):
        start_time = time.time()
        response = self.get_response(request)
        duration = time.time() - start_time
        response_ms = duration * 1000
        user = str(getattr(request, 'user', ''))
        method = str(getattr(request, 'method', '')).upper()
        status_code = str(getattr(response, 'status_code', ''))
        request_path = str(getattr(request, 'path', ''))
        if status_code == '200' and response_ms > 2000:
            logger.info({
                         "message": "*****SLOW RESPONSE****",
                         "path": request_path,
                         "response_time": str(response_ms) + " ms",
                         "method": method,
                         "user": user,
                         "status_code": status_code
                         })
        return response

This logs all APIs which takes greater than 2 seconds to return response. Just add the full path to MIDDLEWARE = ["path.to.APILogMiddleware"] in your settings.py

Answer 7

I found for me that the best and most flexible way was to add logging via a decorator. I simply add the decorator to each of the functions (post, get) that I want to log the request from, as opposed to it being a part of the overal view class. More control over what gets logged. These decorator take the request object passed in (arg[1]) and then logs parts of the request object to a file.

See https://github.com/slogan621/tscharts/commit/39ed479b04b7077f128774d3a203a86d6f68f03e for what amounts to a template for doing this (commit shows changes to settings.py needed to thread the logging into the file logging scheme I have in place, as well as the decorator and example usage).

Answer 8

A few years later and Rhumbix has come out with this library, has anyone tried it?

https://github.com/Rhumbix/django-request-logging

MIDDLEWARE = (
...,
'request_logging.middleware.LoggingMiddleware',
...,
)

And configure logging in your app:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'django.request': {
            'handlers': ['console'],
            'level': 'DEBUG',  # change debug level as appropiate
            'propagate': False,
        },
    },
}

Answer 9

One option is to use django-requestlogs ( https://github.com/Raekkeri/django-requestlogs (I'm the author)). By default it collects request and response data, (among other basic info) and is easy to hook into the Django's logging system.

Answer 10

There is a package already made by jazzband called django-auditlog .

All you have to do is install the package:

pip install django-auditlog

Add the application to your INSTALLED_APPS:

INSTALLED_APPS = [
    ...
    'auditlog',
    ...
]

And run migration:

manage.py migrate

1. If you want automatic logging (Works with DRF), you could:

Add AuditlogMiddleware in your middleware settings

MIDDLEWARE = [
    ...,
    'auditlog.middleware.AuditlogMiddleware',
    ...
]

Registering your model for logging

from auditlog.registry import auditlog
from django.db import models

class MyModel(models.Model):
    pass
    # Model definition goes here

auditlog.register(MyModel)

2. Or by using manual logging

The docs has not been updated for working with DRF but issues like this suggests that It could be implemented. Tested with my own project.