django数据库设计时您将有太多行

Question

I have a django web app with postgres db; the general operation is that every day I have an array of values that need to be stored in one of the tables. There is no foreseeable need to query the values of the array but need to be able to plot the values for a specific day. The problem is that this array is pretty big and if I were to store it in the db, I'd have 60 million rows per year but if I store each row as a blob object, I'd have 60 thousand rows per year.

Is is a good decision to use a blob object to reduce table size when you do not want to query with the row of values? Here are the two options:

option1 : keeping all

group(foreignkey)| parent(foreignkey) | pos(int) | length(int)
  A              |  B                 |  232     |  45
  A              |  B                 |  233     |  45
  A              |  B                 |  234     |  45
  A              |  B                 |  233     |  46
...

option2 : collapsing the array into a blob:

group(fk)| parent(fk) | mean_len(float)| values(blob)
  A      |  B         |    45          |[(pos=232, len=45),...]
...

so I do NOT want to query pos or length but I want to query group or parent. An example of read query that I'm talking about is:

SELECT * FROM "mytable"
LEFT OUTER JOIN "group"
ON ( "group"."id" = "grouptable"."id" )
ORDER BY "pos" DESC LIMIT 100

which is a typical django admin list_view page main query.

Answer 1

I tried loading the data and tried displaying the table in the django admin page without doing any complex query (just a read query). When I get pass 1.5 millions rows, the admin page freezes. All it takes is a some count query on that table to cause the app to crash so I should definitely either keep the data as a blob or not keep it in the db at all and use the filesystem instead.

I want to emphasize that I've used django 1.8 as my test bench so this is not a postgres evaluation but rather a system evaluation with django admin and postgres.