Mongodb and Django

Question

I am currently working with two databases in my django app - mongodb for storing numeric data and mysql for handling relations and providing business domain background.

I have one concern:

First - I am using mongoengine along with django signals and custom tamplate tags. Meaning:

Whenever I call this filter. {{myobject|do_sth:"20"}} the process goes that way:

filter call -> myobject's do_sth property with argument "20" call -> send signal job_done -> receive signal job_done -> create monoengine document -> save it

I dont't know if that's clear but it's just a combination of different techniques - my problem is : How is the efficiency problem in this case ? Has anyone tried that solution or has similar experience ?

Another question: Since, I relatively often use the process described above should I be concerned about number of items in mongodb collection. Let's suppose it will grow up to 10 000 in a week and than we clear the collection - is that fine ?

Answer 1

I'd suggest that is too much logic for a simple template tag and it might be better to do in the view itself - at least thats what djangonauts would argue.

I take it you are outputting the results of the filter - rather than just calling the signals so data is created. If you aren't outputting the results then make the processing offline and non-blocking for the user. I've had great success with celery for long running / potentially blocking tasks (you dont have to complicate your stack as mongodb works great as the queue backend).

MongoDB in production with masses of data across large clusters - so 10,000 documents is a relatively tiny amount of data.