简体   繁体   中英

Optimize SQL query with subquery on same table

I have the following PostgreSQL

        EXPLAIN (ANALYZE, BUFFERS) select h.pk::varchar(255),
            (
                h.fecha at TIME zone 'Mexico/General' at TIME zone 'UTC'
            )::date as fecha
        from historial h
            where (
                h.fecha at TIME zone 'Mexico/General' at TIME zone 'UTC'
            )::date < '2022-07-30'
            and h.del = 0
            and (
                select COUNT(h.pk)
                from historial h
                where h.tipo in (3, 6, 9, 10)
                and h.del = 0
            ) > 10
         order by h.fkcr,
             h.fecha asc

But the historial table has millions of rows and using the ORDER BY clause causes a jump in time from ~300ms to ~6000ms!

Is it possible to optimize it?

This is the EXPLAIN (ANALYZE, BUFFERS) of the comand

I noticed the Sort node is using the Disk to sort it so I tried changing the work_mem to 1GB but it still using the Disk, should the work_mem have worked or I misundertood the function of work_mem

QUERY PLAN                                                                                                                                      |
------------------------------------------------------------------------------------------------------------------------------------------------+
Sort  (cost=215616.65..217123.46 rows=602724 width=40) (actual time=11770.953..13789.063 rows=1810355 loops=1)                                  |
  Sort Key: h.fkcr, h.fecha                                                                                                                     |
  Sort Method: external merge  Disk: 145136kB                                                                                                   |
  Buffers: shared hit=16461 read=49282, temp read=32832 written=32832                                                                           |
  InitPlan 1 (returns $0)                                                                                                                       |
    ->  Aggregate  (cost=48588.68..48588.69 rows=1 width=16) (actual time=479.485..479.486 rows=1 loops=1)                                      |
          Buffers: shared hit=441 read=24000                                                                                                    |
          ->  Bitmap Heap Scan on historial h_1  (cost=3146.86..48088.10 rows=200230 width=16) (actual time=26.643..357.554 rows=186103 loops=1)|
                Recheck Cond: (tipo = ANY ('{3,6,9,10}'::integer[]))                                                                            |
                Filter: (del = 0)                                                                                                               |
                Rows Removed by Filter: 21311                                                                                                   |
                Heap Blocks: exact=23863                                                                                                        |
                Buffers: shared hit=441 read=24000                                                                                              |
                ->  Bitmap Index Scan on idx_tipo  (cost=0.00..3096.80 rows=208414 width=0) (actual time=22.107..22.107 rows=207414 loops=1)    |
                      Index Cond: (tipo = ANY ('{3,6,9,10}'::integer[]))                                                                        |
                      Buffers: shared hit=9 read=569                                                                                            |
  ->  Result  (cost=0.00..92681.53 rows=602724 width=40) (actual time=479.802..7149.979 rows=1810355 loops=1)                                   |
        One-Time Filter: ($0 > 10)                                                                                                              |
        Buffers: shared hit=16453 read=49282                                                                                                    |
        ->  Seq Scan on historial h  (cost=0.00..83640.67 rows=602724 width=40) (actual time=0.293..2781.099 rows=1810355 loops=1)              |
              Filter: ((del = 0) AND ((timezone('UTC'::text, timezone('Mexico/General'::text, fecha)))::date < '2022-07-30'::date))             |
              Rows Removed by Filter: 71719                                                                                                     |
              Buffers: shared hit=16012 read=25282                                                                                              |
Planning time: 1.105 ms                                                                                                                         |
Execution time: 14991.655 ms                                                                                                                    |

For process field casting or using any operations on fields DB cannot use indexes. For this recommended creating expression index. For example:

CREATE INDEX historial_idx1 ON historial ((fecha at TIME zone 'Mexico/General' at TIME zone 'UTC')) 

or

CREATE INDEX test_idx1 ON test ((id::text))

Recommended create index for this fields:

  1. tipo
  2. del
  3. fecha - expression index
  4. fkcr, fecha (together)

And recommended using materialized CTE for subqueries, for example:

with subquery as materialized (
    select COUNT(h.pk)
    from historial h
    where h.tipo in (3, 6, 9, 10)
    and h.del = 0 
) 
select h.pk::varchar(255),
(
    h.fecha at TIME zone 'Mexico/General' at TIME zone 'UTC'
)::date as fecha
from historial h
    where (
        h.fecha at TIME zone 'Mexico/General' at TIME zone 'UTC'
)::date < '2022-07-30'
    and h.del = 0
    and (select * from subquery) > 10
 order by h.fkcr,
     h.fecha asc

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM