Yii2, using a MYSQL table or a .php file

Question

I have millions of texts to save in the database and to show through an application that I am developing with Yii2. A single text can vary from 30 to 500 words, but they are all similar to each other and respect a track where only a few words or values change. To make you understand a text can be

A) Goofy goes shopping on Wednesday at 2.30pm

or

B) Pluto goes shopping on Thursday at 4pm

Instead of saving completely all the texts and words I intend to parameterize them, so I could create a track list

T1 {name} goes shopping on {day} at {hour}

T2 blah blah {param}...

etc...

I guess I have 25-50 tracks.

By doing so, it is not longer necessary to save all the texts and words, so A) becomes a record with track T1 where {name} = Goofy, {day} = Wednesday and {hour} = 2.30pm.

I intend to have a parameters table with this structure

text_id
parameter
value

There will be no text search in the application. The only things that can be searched and filtered are parameters.

To get the best performance, I should save all the tracks in a table or have them all stored in a.php file in an array of key => value and load it during the execution of the application to show each time the texts (of which I know only parameters saved in the db)?

Remembering how Yii handles messages translations through.php files containing arrays, I would like to do the same, but I ask you for confirmation. If so, what would be the optimal management? Insert this array inside the.php model? Or create another.php file to load in the necessary contexts? And where to insert it? In the vendor folder?

Thanks

Answer 1

I'm not sure that you're asking the correct question (about relative performance). I do like your thinking about how to store the data, because it reminds me of how I designed the storage of answers to questionnaires for one of the first online insurance underwriting engines.

I would strongly suggest, given that the majority of your data is going to go into the parameters table, and therefore the overhead of storing the tracks data is minimal, that you store the tracks data in the db too. This will allow you to perform joins directly in the db when necessary - and you will inevitably find reasons to do that. You can then experiment yourself with reading them once into an array or object, on occasions when you would otherwise have to consult the tracks table multiple times in a single action/function.

Your parameters table will need a primary key, or your app will not scale, and if things are going to get really big, and if all of your values, as well as the parameters, are known, then you might also want to consider a further level of abstraction, where your parameters table stores numerical references to the parameters and values, ie it has four columns: unique_id, text_id, parameter_id, value_id, and the parameter_id and value_id are unique values from separate tables. This will give you a huge performance improvement, through optimised indexing. Even if some of the values are free text, you could, as I did, cope with this with an additional column (but requiring extra code).

As I say, the above is all from real-world experience of designing a system that went from nothing to 1 million quotes per day in a couple of years, with each of those quotes requiring retention of 50+ parameter/value pairs similar to yours.