在 django 中存储动态数据

Question

So, in my application I currently am receiving and storing some data,the structure currently is based around this:

• Each application has a number of devices
• Each device has incoming data from its sensors
• Each sensor has a type (eg temperature/humidity/etc)

Initially, I had wanted to find a solution to store the data dynamically into one table for all data, with a model that might look like this:

time = models.DateTimeField()
sensor = models.ForeignKey(Sensor, on_delete=models.CASCADE)
data = ???

Now, each sensor is connected in the models to all the details (including the application, the sensor hardware information and other API-related information for connecting to said sensors), however, for the data I looked up a number of options but I have yet to find one that feels sufficient, some of the options I explored are:

1. Storing the data in a dictionary and storing that in a model as per the answers in Here

2. Having separate tables and somehow calling the models later when storing data using the method here , in addition to some system with inheritance such as discussed here

3. Something similar to this using dynamic fields

I am currently using mySQL, and I saw some other solutions that might work using PostgreSQL for example, but I would like to avoid switching databases if there is another option.

I just feel there must be a better more clean solution for this, as it feels like a straightforward implementation. In a sense most of those implementation either feel too hack-y and inefficient especially considering that I will be storing a large amount of data (A larger number of sensors sending data continuously to the server), or do not allow queries on the sub-data.

Are there any other better implementations for my application? If not, which one of those implementations is best for me to focus on?

Answer 1

The method I ended up using, if anyone faces a similar issue, is as follows:

The main model:

class Sensor(Device):
    sensorName = models.CharField(max_length=50)
    sensorSerial = models.CharField(max_length=50)
    sensorTable = models.CharField(max_length=50)

Each sensor will recieve data, each entry of said data will be saved as follows:

class Entry(models.Model):
    time = models.DateTimeField()
    sensor = models.ForeignKey(Sensor, on_delete=CASCADE)

And each entry will contain a number of fields which are unknown, depending on what kind of sensor it was, so a third class for the DATA is included and used a pseudo-dictionary:

class Data(models.Model):
    entry = models.ForeignKey(Entry, on_delete=CASCADE)
    key = models.CharField(max_length=50)
    value = models.CharField(max_length=50)

I am still investigating how to push the effieciency higher, but this is definitely much faster than looping through entries while decoding XML/JSON over 100s of thousands of entries.

I will update the answer with any breakthroughs or updates I find.