Will pg_dump and pg_restore of PostgreSQL affect the buffer cache and kernel file system cache?

Question

May I query about the buffer cache behavior of PostgreSQL during pg_dump and pg_restore ?

As we know, PostgreSQL has a buffer cache to cache the recent working set, and Linux also has its file system level cache.

When we use pg_dump to backup the database, would the backup operation affect the PostgreSQL buffer cache and the system file cache?

And what about the pg_restore operation?

Answer 1

Since these operations read or write files on the machine, they will certainly affect the kernel's file system cache, potentially blowing out some data that were previously cached.

The same is true for the PostgreSQL shared buffers, although there is an optimization that avoids overwriting all shared buffers during a large sequential scan: if a table is bigger than a quarter on shared buffers, a ring buffer of 256 kB will be used rather that eliminating a major part of the cache.

See the following quotes from src/backend/access/heap/heapam.c and src/backend/storage/buffer/README :

    /*
     * If the table is large relative to NBuffers, use a bulk-read access
     * strategy and enable synchronized scanning (see syncscan.c).  Although
     * the thresholds for these features could be different, we make them the
     * same so that there are only two behaviors to tune rather than four.
     * (However, some callers need to be able to disable one or both of these
     * behaviors, independently of the size of the table; also there is a GUC
     * variable that can disable synchronized scanning.)
     *
     * Note that table_block_parallelscan_initialize has a very similar test;
     * if you change this, consider changing that one, too.
     */
    if (!RelationUsesLocalBuffers(scan->rs_base.rs_rd) &&
        scan->rs_nblocks > NBuffers / 4)
    {
        allow_strat = (scan->rs_base.rs_flags & SO_ALLOW_STRAT) != 0;
        allow_sync = (scan->rs_base.rs_flags & SO_ALLOW_SYNC) != 0;
    }
    else
        allow_strat = allow_sync = false;

For sequential scans, a 256KB ring is used. That's small enough to fit in L2
cache, which makes transferring pages from OS cache to shared buffer cache
efficient.  Even less would often be enough, but the ring must be big enough
to accommodate all pages in the scan that are pinned concurrently.  256KB
should also be enough to leave a small cache trail for other backends to
join in a synchronized seq scan.  If a ring buffer is dirtied and its LSN
updated, we would normally have to write and flush WAL before we could
re-use the buffer; in this case we instead discard the buffer from the ring
and (later) choose a replacement using the normal clock-sweep algorithm.
Hence this strategy works best for scans that are read-only (or at worst
update hint bits).  In a scan that modifies every page in the scan, like a
bulk UPDATE or DELETE, the buffers in the ring will always be dirtied and
the ring strategy effectively degrades to the normal strategy.

As the README indicates, that strategy is probably not very effective for bulk writes.

Still, a pg_dump or pg_restore will affect many tables, so you can expect that it will blow out a significant portion of shared buffers.