Is the usage of scala collection equivalents in slick operation is harmful?

Question

I have tables table_a and table_b in my database and they are mapped in slick with TableQuery Objects. I need to copy a restricted set of data from table_a to table_b .

Let the table query objects be tableQueryA and tableQueryB . The logic for filtering and copying data is complex. So I think of doing scala collection equivalent of table query object in a for yield and treat them as normal collections. But Everything happens in one transaction. The code looks something like this.

for {
    collA <- tableQueryA.filter(.....something....).result
    collB <- tableQueryB.filter(.....somethingElse.....).result
    ...... do something with collA and collB
    }
 yield ...something

Is there a harm doing this way, ie handling as scala collections and processing them? I am using slick 3.2

Answer 1

By doing two separate tableQueryX.filter().result , you'll be executing two separate query to the database. You could replace it with one query that joins two tables.

It's hard to say what is the better approach in term of performance as it depends on amount of filter or where clauses and what kind of indexes are used by the database to fulfil those. If you need a top notch performance, try both approaches and pick one that is the fastest.

If both of your queries yield big amount of data, you need to consider memory usage for your application too because all data is loaded before scala collection api can be used.

Answer 2

只要数据较少，我看不到任何危害-但最好在数据库级别过滤掉数据，以避免任何潜在的内存不足错误。