What is the most efficient way to populate data to TListView using TFDQuery triggered by another TListView on selected item as parameter in FireDAC

Question

I have a million records in one of my database table which my FDQuery is pointed to. Here's my SQL query:

SELECT DISTINCT variant FROM item_var
JOIN item ON item_var.item_id = item.list_id
WHERE item.brand = :itemselected
ORDER BY variant                              //I did not put a limit because the data here very limited say maximum  10 only

The parameter ":itemselected" is to be triggered by my TListView1 selected item.

The result of this query is populate to be populated to my TListView2 with this code:

if dmMain_u.dmMain.qryVariant.RecordCount >= 0 then
  begin
    with dmMain_u.dmMain do
    begin
      qryVariant.Active := False;
      qryVariant.Params.ParamByName('itemselected').AsString := lsvItems.Items[lsvItems.ItemIndex].Text;
      qryVariant.Active := True;
    end;
  end;

I am using livebindings for both TListViews and my database is SQLite embedded local.

The above codes seems to taking a while to populate the data to my TListView2 (like 5 seconds). I am hoping that you guys have better options to share. Perhaps, my codes are not efficient or optimize enough.

Answer 1

It's a general rule of database design that any column that are used to access db rows (both to look rows up and join tables together) should be indexed. This is so that the server (or server process in the case of Sqllite) can find the relevant rows without having to examine all the rows in the table (aka a 'table scan'), which coud be hugely inefficient where only a small proportion of the rows in the table are relevant to (ie match) the requirenents of the task at hand. Without suitable indices, there will likely be performance problems just like the one you are asking about.

Most db engines, include a 'query optimiser' which tries to find the most efficient way to perform a query and one of the main strategies which they use is to look up matching rows using an index against the table(s) involved, as this can greatly reduce the number of rows which need to be read from the table. In some cases, known as 'covered queries', the db-engine does not even need to read the table rows at all, because it can retrieve all the information it needs (including the column data requested by the query) from an index on the table.

This is all explained with reference specifically to Sqlite in Section 1.7, 'Covering indices' of this article: sqlite.org/queryplanner.html