Allowed memory size exhausted + poor performance

Question

I have Requests table in my MySQL database (Yii project), which contains all visits to the main page of my website with

fields: `id`, `partner_id` (relation to `Partners` table), `date` 

date with current timestamp.

and id with id of a visitor.

The problem is that there is about 1.5 millions rows in the Requests table and when I try to expose this data somewhere, it shows poor performance or even error from title.

For example, I've got the list of partners and column called "All requests" where I'm viewing all requests with partner_id equal to this partner's id. There could be something like 500 or 1000 requests for every partner, so, I believe, my SQL query is very heavy.

Could you please give some advices about how to make it more flexible?

I still need this data from Requests for commercial calculations, but I can't find out how to avoid this huge amount of data and poor performance.

Answer 1

Don't execute request 500 or 1000 on a same time. If you do so then it will gives you error or time out error.

Better to put some logic (kind of forloop or foreach) statement and send request in bunch of 10-20 (but not more than that). If you follow this then it will gives you result properly.

Secondly, I don't know about your front end, but suppose you use DotNet then put a call with "async" "await" keyword in your method which handle this kind of task easily.

If you need some more details on this then please revert.