'Simple' mysql query takes 30 seconds to load

Question

I'm creating a pretty basic MySQL query using a Laravel Eloquent Model. I've three database tables: posts , tags and post_tag (pivot table). I want to get and paginated the most used tags. The query is working but takes around 30 seconds to load. The tags table has ~9k records and the post_tag table has ~26k records. But in the near future their will be more than 1 million records in the post_tag table.

The code is as following:

$tags = Tag::leftJoin('post_tag', 'tags.id', '=', 'post_tag.tag_id')
    ->select(\DB::raw('post_tag.tag_id, tags.content, COUNT(`tag_id`) AS `occurrence`'))
    ->groupBy('tag_id', 'content')
    ->orderBy('occurrence', 'desc')
    ->take(15)
    ->paginate(request('per_page'));

Without the pagination the query takes around 20 seconds to load. I don't often use group by and join queries, but my guesses are that the group by is asking a lot of time.

I'v surfed on Stackoverflow and Google, but without any luck. Any suggestions on what to try? Is an index going to much difference?

Answer 1

IMHO, it is better to define the model relationships after created corresponding tables, indexes and foreign keys and leverage Laravel Eloquent to do the job. I almost never use query builder in any of my Laravel projects to create queries, I instead, as I said, spend time and effort to design the database tables and corresponding models and their relationships, that has proved to me to end up with cleaner code that performs better. Again, it is just my personal point of view. Having said that, I am showing a piece of code that produce the outcome you want:

Assuming having tables: posts, tags and post_tag. The later having post_id and tag_id fields declared as index, with foreign_key to their related table.

Defining model relatioships:

On Tag model:

public function posts()
{
    return $this->belongsToMany(Post::class);
}

On Post model:

public function tags()
{
   return $this->belongsToMany(Tag::class);
}

On controller:

$tags = \App\Tag::withCount('posts')->orderBy('posts_count', 'DESC')->paginate(10);

Realize that the amount of posts a tag has is indeed its occurrence.

Related topic on Laravel doc: https://laravel.com/docs/5.5/eloquent-relationships#counting-related-models

Answer 2

You didn't post sample data, so I can't do any tests.

It's just an idea, so if it doesn't work I will delete the answer.

I was thinking you could try (and check for performances) to change from your query (as far as I understood reading your code)

(Updated: Added LIMIT clause thanks to RolandStarke)

SELECT A.TAG_ID, B.CONTENT, COUNT(*) AS OCCURRENCE
FROM POST_TAG A
INNER JOIN TAGS B ON A.TAG_ID= B.ID
GROUP BY A.TAG_ID, B.CONTENT
ORDER BY OCCURENCE DESC
LIMIT 15;

to this one:

SELECT A.TAG_ID, B.CONTENT, OCCURRENCE
FROM (SELECT TAG_ID, COUNT(*) AS OCCURENCE 
      FROM POST_TAG 
      GROUP BY TAG_ID 
      ORDER BY OCCURRENCE DESC LIMIT 15) A
INNER JOIN TAGS B ON A.TAG_ID= B.ID
ORDER BY OCCURENCE DESC;

Pls let me know