Making SQL query faster

Question

I currently have an SQL query that is taking a LONG time to load, wondered if anyone could help me? I'm fairly new with MySQL.

This is the query:

SELECT applicationid 
FROM logs 
WHERE action = 'VIEWED' 
AND userid = '$user' 
AND date >= '$dateminus7' 
AND applicationid NOT IN (
     SELECT applicationid 
     FROM logs 
     WHERE userid = '$user' 
     AND date >= '$dateminus7' 
     AND (action LIKE 'SUBMITTED NOTE%' OR action LIKE 'CHANGED STATUS%')
)

Basically looking through some databases to find users that are not leaving notes when they access customer accounts (very naughty). There are around 30,000 records per week in the logs database, which is obviously a factor, but right now the query runs for an hour and still doesn't complete (times out, 404 error on PHP page).

Any info needed just ask, I'd appreciate any tips or pointers.

EDIT: EXPLAIN results http://i.imgur.com/h9bZBe3.png

Answer 1

A composite index on (userid, date) should be sufficient to speed up both queries here. You can also try (userid, date, action) or (action, userid, date) . Depending on which column has more unique values, one index might be more effective than another.

Note that the query planner might not be able to optimize the subquery, and it will potentially execute the subquery once for each candidate row in the outer query. The cost of running the inner query multiple times could be contributing to the performance problem. Consider trying a join instead, and see if you get better performance:

SELECT la.applicationid 
FROM logs la

LEFT OUTER JOIN logs lb
    ON la.applicationid = lb.applicationid
    AND lb.userid = '$user' 
    AND lb.date >= '$dateminus7' 
    AND (lb.action LIKE 'SUBMITTED NOTE%' OR lb.action LIKE 'CHANGED STATUS%')

WHERE la.action = 'VIEWED' 
AND la.userid = '$user' 
AND la.date >= '$dateminus7' 
AND lb.applicationid IS NULL;

Answer 2

I've found that MySQL doesn't optimize IN and NOT IN queries very well. Replacing them with a JOIN usually improves things. For NOT IN you have to use a LEFT OUTER JOIN :

SELECT l1.applicationid
FROM logs l1
LEFT OUTER JOIN (SELECT applicationid 
     FROM logs 
     WHERE userid = '$user' 
     AND date >= '$dateminus7' 
     AND (action LIKE 'SUBMITTED NOTE%' OR action LIKE 'CHANGED STATUS%')) l2
ON l1.applicationid = l2.applicationid
WHERE l1.action = 'VIEWED' 
AND l1.userid = '$user' 
AND l1.date >= '$dateminus7'
AND l2.applicationid IS NULL

Answer 3

You have two "action='XXX'" in two querys(main one and the subquery) and they dont ever have any records in Common I think. In other words, if you are looking for some applications with the action "VIEWED". You dont need to use the query:

SELECT applicationid FROM logs WHERE userid = '$user' AND date >= '$dateminus7' AND (action LIKE 'SUBMITTED NOTE%' OR action LIKE 'CHANGED STATUS%')

Not sure if I missed something and whether this will help. I will follow this.