Using Django with legacy databases containing multiple tables

Question

I have an old application written in php that provides access to large datasets that are organized into 37 different databases, with each database containing a number of tables. Each table represents data from a different piece of equipment, and each database represents the data from a single time period, so the structure is as follows:

Database_Time_Period_one   
 table machine 1
 table machine 2
 table machine 3
 and so on...

Database_Time_Period_two
 table machine 1
 table machine 2
 table machine 3
 table machine 4
 and so on...

The tables vary in length but typically contain between 500,000 and 1,000,000 rows of data.

The application needs to be extended and needs a new front end, and am looking at reworking it using Django, but I am having a conceptual problem which I am hoping someone can help with. The php code is very simple - the user selects the time period and table of interest in the front end, and this is used to build a MySQL SELECT query that retrieves the relevant data which is then stored in an array object. The query is constructed in a single line of code that essentially uses a variable taken from the front end to identify the database, and another for the table. The naming of the databases and tables follows a pattern that includes a sequential numeric component and this simplifies the construction of the query.

As I understand things from my initial forays into Django, a django model represents a single table from a single database, and so to be able to access all of my tables across the different databases, I would need to create a model for each table. This would mean having around 200 models. Also, I would need some way to tell django which database and which table in that database the model should be linked to. Finally, I would have to be able to select in code which model to use for any given query. I have looked into this and it seems that it would be possible to do this, but it would mean creating a large number of routers, and a lot of code compared to the relatively simple php code.

Is it possible to have simple code that would essentially tell django: "Go to database x, look at table y, and select the following columns" even if this mean circumventing the use of a model and just reading the data into an array?

I have searched online for a solution to this, but can't find an example that appears to address this situation, and while the django documentation does explain the use of multiple databases, it does not seem to provide a simple way to access the databases that I have without creating what looks like a very large and cumbersome body of code, and the explanation of using raw queries still seems to rely on the 'one model per table' paradigm. I am sure that my difficulty is due to my limited experience with Django and that there is a way to do what I want, but I at the moment I am stuck.

It would be very helpful is someone with considerably more django experience than I have could point me in the right direction, or tell me if this looks like a situation that django is just not suited to.

Answer 1

Do you know about django's inspectdb management command:

$ python manage.py inspectdb > models.py

This will produce models for you from which you can use the django orm to do whatever you need to do with your data.

http://docs.djangoproject.com/en/dev/ref/django-admin/#inspectdb

https://docs.djangoproject.com/en/1.7/howto/legacy-databases/

You can write sql code directly and not make models if you think that is easier. Look here: https://docs.djangoproject.com/en/1.7/topics/db/sql/#executing-custom-sql-directly