Android SQLITE query slow on large number of rows

Question

I have 2 tables and the query I need runs very slow. For exemplification, I show two sample tables with the relevant fields.

Table MAIN
      MAIN.USERID text
      MAIN.LOGGED_USERID text
      MAIN.other text fields not relevant to query
Index on USERID, LOGGED_USERID

Table LOGS
      LOGS.CREATED int
      LOGS.USERID text
      LOGS.LOGGED_USERID text
      LOGS.TYPE int

Index on USERID,LOGGED_USERID,TYPE

I need to get the most recent date of a USER/LOGGED_USER from the LOGS table. The query I need to run is similar to:

Select m.some fields,
   (Select l.CREATED 
    from LOGS l 
    where l.USERID=u.USERID and l.LOGGED_USERID=m.LOGGED_USERID and TYPE=1 
    order by l.CREATED desc limit 1) as LAST_DATE
from MAIN m
WHERE some fields not relevant and an EXIST command from other table
ORDER BY some fields not relevant

While the query runs fine for a limited number of rows, when having 150.000 rows on each table, the query actually never seems to finish, or at least I haven't waited that long for it. The performance drop comes from the LAST_DATE query, as without it, it's instant Probably the indexes I have are not used?

Answer 1

You solution execute the sub-query for each row.

Using a join, this will be executed only once then join both tables.

Here is the query to get the last created LOGS per user_id and logger_userid

Select max(CREATED), LOGGED_USERID, USERID
       from LOGS l 
       where TYPE=1 
       group by LOGGED_USERID, USERID

And to use it in a join, simply use an alias :

Select m.some fields,
   l.CREATED as LAST_DATE
   from MAIN m
   inner join (Select max(CREATED), LOGGED_USERID, USERID
       from LOGS l 
       where TYPE=1 
       group by LOGGED_USERID, USERID
   ) l on l.USERID = M.USERID ANd l.LOGGED_USERID = M.LOGGED_USERID.

NOTE : I am note sure of SQLite notation, I've worked on Sybase for too long ..

This still needs the where and order condition of course, this is just to correct the select .. from ... part

Answer 2

As you do not show your code, I think you are taking too much work from the UI thread. So by taking this assumption you should use Loaders. In your case Cursor Loaders . Loaders basically run on separate thread and avoids slow UI. See the link of Loaders explained: Loaders in Android