Tracing requests of users by logging their actions to DB in django

Question

I want to trace user's actions in my web site by logging their requests to database as plain text in Django.

I consider to write a custom decorator and place it to every view that I want to trace. However, I have some troubles in my design. First of all, is such logging mecahinsm reasonable or because of my log table will be enlarging rapidly it causes some preformance problems ? Secondly, how should be my log table's design ?

I want to keep keywords if the user call search view or keep the item's id if the user call details of item view. Besides, IP addresses of user's should be kept but how can I seperate users if they connect via single IP address as in many companies.

I am glad to explain in detail if you think my question is unclear.

Thanks

Answer 1

I wouldn't do that. If this is a production service then you've got a proper web server running in front of it, right? Apache, or nginx or something. That can do logging, and can do it well, and can write to a form that won't bloat your database, and there's a wealth of analytical tools for log analysis.

You are going to have to duplicate a lot of that functionality in your decorator, such as when you want to switch it on or off, or change the log level. The only thing you'll get by doing it all in django is the possibility of ultra-fine control, such as only logging views of blog posts with id numbers greater than X or something. But generally you'd not want that level of detail, and you'd log everything and do any stripping at the analysis phase. You've not given any reason currently why you need to do it from Django.

If you really want it in a RDBMS, reading an apache log file into Postgres or MySQL or one of those expensive ones is fairly trivial.

Answer 2

One thing you should keep in mind is that SQL databases don't offer you a very good writing performance (in comparison with reading), so if you are experiencing heavy loads you should probably look for a better in-memory solution (eg. some key-value-store like redis ).

But keep in mind, that, especially if you would use a non-sql solution you should be aware what you want to do with the collected data (just display something like a 'log' or do some more in-deep searching/querying on the data).

If you want to identify different users from the same IP address you should probably look for a cookie-based solution (if you are using django's session framework the session's are per default identified through a cookie - so you could just simply use sessions). Another solution could be doing the logging 'asynchronously' via javascript after the page has loaded in the browser (which could give you more possibilities in identifying the user and avoid additional load when generating the page).