Optimize query with big-table-join

Question

I have one big table.

Let's call it 'unicorns'.

And second table 'farmers'.

Structure:

farmers

• id
• name
• ... some more fields

unicorns

• id
• farmer_id
• ... some more fields

And there is query:

SELECT
  f.id,
  f.name,
  COUNT(*)
FROM
  farmers f 
LEFT JOIN unicorns u 
ON u.farmer_id = f.id 
GROUP BY f.id

Of course tables are really named not this way and have more fields etc. But I have left only the most important things in my example.

The problem is that the system grows, and there are too many unicorns. Millions.

And this query (like-this) is executed on farmers list page.

And page is loading not so fast as before, because we join a multi-million table each time we load page.

Problem is:

• We really need to display each farmer's unicorns count in the list.
• We need to improve page load speed.

If you would need to optimize this, how would you achieve this result?

What would you recommend?

PS

Personally I think I need to exclude big-table-join from the main query, calculate unicorns counts separately and store them in the cache storage, recalculate them time after time. Maybe there is the best way, so I wanted to hear someone else's opinion.

Answer 1

I would just have an extra column on the farmer's table for NumberUnicorns and store it there. Then, via some database triggers, when a record is added to the unicorn table, it just updates the farmers table with the respective count from the new record. Also consider updating the count if unicorn records are deleted. Then, your query is immediately from the farmers table -- done.