Postgres 查询有时需要很长时间

Question

I have implemented a FTS engine in my website using GIN tsvector and it works quite well, but there are a few times when it seems to take a very long time, for no specific reason. I am copying the output of the EXPLAIN ANALYE command below:

sitedb=# EXPLAIN ANALYZE SELECT id, title FROM post_1 WHERE search_vector @@   to_tsquery('quantum');
                                                      QUERY PLAN                                                           
-------------------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on post_1  (cost=315.68..105654.80 rows=32443 width=106) (actual   time=76.963..17281.184 rows=31925 loops=1)
   Recheck Cond: (search_vector @@ to_tsquery('quantum'::text))
   Heap Blocks: exact=29259
   ->  Bitmap Index Scan on index1_idx  (cost=0.00..307.57 rows=32443 width=0) (actual time=60.208..60.209 rows=31925 loops=1)
     Index Cond: (search_vector @@ to_tsquery('quantum'::text))
 Planning Time: 47.648 ms
 Execution Time: 17308.511 ms
(7 rows)

I thought at some point that changing work_mem would help. I have set it up to 86MB and it is still the same.

The weird thing is that if I re-run the same command right after, it is much faster. See below:

sitedb=# EXPLAIN ANALYZE SELECT id, title FROM post_1 WHERE search_vector @@ to_tsquery('quantum');
                                                      QUERY PLAN                                                           
-------------------------------------------------------------------------------------------------------------------------------
 Bitmap Heap Scan on post_1  (cost=315.68..105654.80 rows=32443 width=106) (actual time=44.542..495.525 rows=31925 loops=1)
   Recheck Cond: (search_vector @@ to_tsquery('quantum'::text))
   Heap Blocks: exact=29259
   ->  Bitmap Index Scan on index1_idx  (cost=0.00..307.57 rows=32443 width=0) (actual time=29.256..29.256 rows=31925 loops=1)
     Index Cond: (search_vector @@ to_tsquery('quantum'::text))
 Planning Time: 0.597 ms
 Execution Time: 502.296 ms
(7 rows)

Would anyone have an idea?

Thank you very much.

Answer 1

It is probably a cold cache. Eg it had to read 29,259 pages and few of them were already in memory. The 2nd time you run it, they are in memory, so it is faster. You could help confirm this by doing EXPLAIN (ANALYZE, BUFFERS) after turning track_io_timing on.

You can increase effective_io_concurrency so that PostgreSQL will have multiple IO requests outstanding at once. How effective this is will depend on your IO hardware. It should be more effective on striped RAID or JBOD, for example.

If your cache was cold because you recently restarted, well, don't restart very often, or use pg_prewarm to warm up the cache when you do. If it can't stay in cache because your frequently-used data is too big for it to stay in memory, then get more RAM, or get faster disks (like SSD, of if they already are SSD then get better ones).