Laravel 查询优化 (SQL)

Question

My user_level database structure is

| user_id | level |
|       3 |     F |
|       4 |    13 |
|      21 |     2 |
|      24 |     2 |
|      33 |     3 |
|      34 |   12+ |

I have another table users

    |      id | school_id |
    |       3 |         3 |
    |       4 |         4 |
    |      21 |         2 |
    |      24 |         2 |
    |      33 |         3 |
    |      34 |         1 |

What I have to achieve is that, I will have to update the level of each user based on a certain predefined condition. However, my users table is really huge with thousands of records.

At one instance, I only update the user_level records for a particular school. Say for school_id = 3 , I fetch all the users and their associated levels, and then increase the value of level by 1 for those users (F becomes 1, 12+ is deleted, and all other numbers are increased by 1).

When I use a loop to loop through the users, match their user_id and then update the record, it will be thousands of queries. That is slowing down the entire application as well as causing it to crash.

One ideal thing would be laravel transactions, but I have doubts if it optimises the time. I tested it in a simple query with around 6000 records, and it was working fine. But for some reason, it doesnt work that good with the records that I have.

Just looking some recommendation on any other query optimization techniques.

UPDATE

I implemented a solution, where I would group all the records based on the level (using laravel collections), and then I would only have to issue 13 update queries as compared to hundreds/thousands now.

$students = Users::where('school_id', 21)->get();
$groupedStudents = $students->groupBy('level');
foreach ($groupedStudents  as $key => $value) :
        $studentIDs = $value->pluck('id');
        // condition to check and get the new value to update
        // i have used switch cases to identify what the next level should be ($NexLevel)
       UserLevel::whereIn('userId', $studentIDs)->update(["level" => $nextLevel]);
endforeach;

I am still looking for other possible options.

Answer 1

First defined a relationship in your model, like:

In UserLevel model:

public function user() {
      return $this->belongsTo(\App\UserLevel::class);
}

And you can just update the level without 12+ level's query, only by one query , and delete all 12+ level by one query .

UserLevel::where('level', '<=', 12)->whereHas('user', function($user) {
    $user->where('school_id', 3);
})->update(['level' => DB::raw("IF(level = 'F', 1, level+1)")]);

UserLevel::whereHas('user', function($user) {
    $user->where('school_id', 3);
})->where('level', '>', 12)->delete();

If your datas is too huge. you can also use chunk to split them for reduce memory consumption.

like this:

UserLevel::where('level', '<=', 12)->whereHas('user', function($user) {
    $user->where('school_id', 3);
})->chunk(5000, function($user_levels) {
    $user_levels->update(['level' => DB::raw("IF(level = 'F', 1, level+1)")]);
});


UserLevel::whereHas('user', function($user) {
    $user->where('school_id', 3);
})->where('level', '>', 12)->delete();

Answer 2

There are many ways to optimise query which are as below:

=> At this situation, you better use Cache and doesn't need to hit the database for every time. It can reduce the processing time and have better performance.

=> Indexing various tables, this brought the time down to 10 seconds, again far too long

=> You can also use select in query which reduces the time to fetch all columns data