使用 Eloquent whereNotIn 时查询非常慢

Question

I have a Question model from very large table of questions (600,000 records), with relation to Customer , Answer and Product models. Relations are irrelevant to this question but I mentioned them to clarify I need to use Eloquent. When I call Question::with('customer')->get(); it runs smoothly and fast.

But there is another table in which I have question_id s of all questions which should not be shown (for specific reasons).

I tried this code:

    // omitted product ids, about 95,000 records
    $question_ids_in_product = DB::table('question_to_product')
                  ->pluck('product_id')->all();
    $questions =  Question::with('customer')
                  ->whereNotIn('product_id', $question_ids_in_product)
                  ->paginate($perPage)->get();

It takes so much time and shows this error: SQLSTATE[HY000]: General error: 1390 Prepared statement contains too many placeholders

and sometimes Fatal error: Maximum execution time of 30 seconds exceeded

When I run it with plain sql query:

SELECT * FROM questions LEFT JOIN customers USING (customer_id) WHERE question_id NOT IN (SELECT question_id FROM question_to_product)

it takes only 80 milliseconds

How can I use Eloquent in this situation?

Answer 1

You can use whereRaw method:

$questions =  Question::with('customer')
              ->whereRaw('question_id NOT IN (SELECT question_id FROM question_to_product)')
              ->paginate($perPage)->get();

But ideally as you found out this is a better sollution:

Question::with('customer')->whereNotIn('question_id', 
    function ($query) { 
        $query->from('question_to_product') ->select('question_id'); 
    }
);

Difference?

1. When you will migrate your database to another database the whereRaw might not work as you put in raw statements. That is why we have Eloquent ORM which handles these transitions and build the appropriate queries to run.

2. No performance impact because the SQL is the same (for MySQL )

PS: For better debugging try installing this debug bar

Answer 2

refer from https://laravel.com/docs/5.4/queries#where-clauses

$users = DB::table('questions')
            ->leftJoin('customers', 'curtomer.id', '=', 'question.user_id')
            ->whereNotIn('question_id', [1, 2, 3])
            ->get();

Answer 3

It'll work 100%. When you query getting longer to response like more than 30 seconds when you are using whereNotIn. Use this Query Syntax.

 $order = Order::on($databaseCredentials['database'])
->whereRaw('orders_id NOT IN (SELECT orders_id FROM orders)')
->skip($page)
->take(10)
->orderBy('orders.updated_at', 'ASC')
->paginate(10);