Skip to content

Improve VACUUM instrumentation and performance logging - #2078

Open
Alena0704 wants to merge 13 commits into
apache:REL_2_STABLEfrom
Alena0704:vacuum_instrumentation_rel2
Open

Improve VACUUM instrumentation and performance logging#2078
Alena0704 wants to merge 13 commits into
apache:REL_2_STABLEfrom
Alena0704:vacuum_instrumentation_rel2

Conversation

@Alena0704

@Alena0704 Alena0704 commented Oct 5, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

What does this PR do?

Make VACUUM logs more useful for explaining expensive maintenance runs:
how much work was done, what could not be cleaned up, why an aggressive
scan was needed, and how much time was spent in cost-based delays.

  • Count heap pages with unremovable dead tuples, pages where tuples were
    frozen, and pages marked all-visible. Include these measurements in
    VACUUM VERBOSE and autovacuum logs.
  • Distinguish scans required by transaction or multixact freeze age from
    scans forced only by DISABLE_PAGE_SKIPPING.
  • Measure cost-based VACUUM delay, excluding ANALYZE sampling. Add delay
    information to heap and index logs and removed index tuples to the
    per-index autovacuum summary.
  • Measure AO row/column vacuum phases, remaining hidden tuples and index
    cleanup delays. Keep compaction tuple and byte counters 64-bit.
  • Correct SP-GiST's count of newly deleted pages.

Also collect visibility-map clearings and cumulative maintenance times in
ordinary relation/database statistics. Keep manual VACUUM, autovacuum,
ANALYZE and autoanalyze timings separate, and avoid adding index time twice
to database totals. SQL access to these counters is supplied by the
follow-up vacuum_stats PR.

This is the first part of the vacuum statistics series, ending at
next PR. PostgreSQL backports are adapted
to Cloudberry's PostgreSQL 14 statistics collector.

Type of Change

  • New feature
  • Bug fix

Impact

Cost-delay measurement requires track_cost_delay_timing = on; it is off
by default and can be enabled for a session. Cumulative collection follows
track_counts. Elapsed times include delays and other waits, so subtracting
cost delay does not produce CPU time.

Checklist

@Alena0704 Alena0704 mentioned this pull request Oct 5, 2026
Open
3 tasks done
@Alena0704 Alena0704 added the type: Enhancement New feature or request, ideas label Oct 5, 2026
@Alena0704
Alena0704 requested a review from leborchuk October 5, 2026 16:20
@Alena0704
Alena0704 force-pushed the vacuum_instrumentation_rel2 branch from 54dc3de to ccadeff Compare October 6, 2026 10:47
nathan-bossart and others added 12 commits October 7, 2026 23:23
@nathan-bossart @Alena0704
Add is_analyze parameter to vacuum_delay_point().
58f7c8d 
This function is used in both vacuum and analyze code paths, and a
follow-up commit will require distinguishing between the two.  This
commit forces callers to specify whether they are in a vacuum or
analyze path, but it does not use that information for anything
yet.

Author: Nathan Bossart <nathandbossart@gmail.com>
Co-authored-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/ZmaXmWDL829fzAVX%40ip-10-97-1-34.eu-west-3.compute.internal

Backported from PostgreSQL 18 (commit e5b0b0c)
as a prerequisite of the vacuum statistics series.  Cloudberry adaptation:
the append-optimized and AOCS sample-row acquisition and compaction call
vacuum_delay_point() too; the former pass is_analyze = true, the latter
false.

(cherry picked from commit e07fe98)
(cherry picked from commit ffe16da)
@nathan-bossart @Alena0704
Measure cost-based vacuum delay
e0f04bc 
Introduce the process-local VacuumDelayTime accumulator and measure time
spent in cost-based VACUUM delays, in microseconds. Gate the measurement
with track_cost_delay_timing, disabled by default. Consumers take
differences around their operations to attribute the measured delay.

Use is_analyze to exclude ANALYZE sampling and statistics computation from
the vacuum delay accumulator. Cost-based throttling and interrupt checks
continue to apply to both operations.

Port the delay measurement from PostgreSQL 18 without its progress-view
changes, so this commit needs no catalog update. Omit the progress slots,
progress-increment helper and parallel progress reporting. The accumulator
is process-local and Cloudberry does not enable parallel vacuum. Register
the GUC in the unsynchronized list.

Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Co-authored-by: Nathan Bossart <nathandbossart@gmail.com>
Co-authored-by: Alena Rybakina <alenka.rybakina@gmail.com>
Discussion: https://postgr.es/m/ZmaXmWDL829fzAVX%40ip-10-97-1-34.eu-west-3.compute.internal

Backported from PostgreSQL 18 (commit bb8dff9).
(cherry picked from commit 79c8bbc)
@Alena0704 @danolivo @zubkov-andrei
Track table visibility map stability
2eec30d 
Collect visible_page_marks_cleared and frozen_page_marks_cleared.
Count transitions of heap visibility-map bits from
set to clear during normal backend activity. Further modifications of
a page whose bit is already clear do not increment that counter. Count
clearings even if the modifying transaction subsequently rolls back.

Comparing the all-visible clearing rate with DML volume helps identify
modifications spread over pages vacuum previously marked all-visible.
Index-only scans must visit the heap for those pages until visibility
is re-established. The all-frozen counter shows how often pages lose
the property that allows aggressive vacuum to skip them. Clearing this
bit does not unfreeze tuples that are already frozen.

Adapt the v44 VM stability patch to Cloudberry's statistics collector.
Deliver the counters with ordinary table-statistics messages and
accumulate both relation and database totals. Collection follows
track_counts. These are DML-driven visibility-map transitions rather than
work performed by VACUUM. SQL access and TAP coverage follow in the
separate vacuum_stats extension commit; this commit does not change the
system catalog.

Count the cleared bits in visibilitymap_clear(), which already takes
a Relation on this branch. Skip relations without a statistics entry,
including fake relations used during recovery.

Authors: Alena Rybakina <lena.ribackina@yandex.ru>,
         Andrei Lepikhov <lepihov@gmail.com> (@danolivo),
         Andrei Zubkov <zubkov@moonset.ru> (@zubkov-andrei)
Based-on: https://www.postgresql.org/message-id/attachment/204710/v44-0009-Track-table-VM-stability.patch

Co-authored-by: Andrei Lepikhov <lepihov@gmail.com>
Co-authored-by: Andrei Zubkov <zubkov@moonset.ru>
@Alena0704
Measure heap and index vacuum work per call
1ffb5ec 
Assemble backend-local tuple and page measurements and take per-call
snapshots of the existing index results. Access methods accumulate results
across passes, so retain each pass's delta without counting earlier work
again.

Keep SP-GiST's newly deleted page count cumulative within one vacuum,
without counting pages that were already empty on entry. Assigning all
deleted pages to pages_newly_deleted would count empty pages again on
subsequent scans.
@Alena0704
Count vacuum pages with unremovable dead tuples
f6072fe 
Introduce dead_pages in LVRelState and PgStat_VacuumStats. Increment it
for each heap page that retains dead tuples after pruning. For index
cleanup, derive the count from deleted pages not yet reusable.

Collector delivery and SQL exposure follow in the reporting commit.

Include these per-operation measurements in VACUUM VERBOSE output.
@Alena0704
Count heap pages where vacuum freezes tuples
b35b29a 
Introduce pages_frozen in LVRelState and PgStat_VacuumStats. Increment
the counter when vacuum executes freezing work on a heap page and
include it in the backend-local measurements. Collector reporting
and SQL exposure follow separately.

Include these per-operation measurements in VACUUM VERBOSE output.
@Alena0704
Count heap pages marked all-visible by vacuum
7c8f239 
Introduce pages_all_visible in LVRelState and PgStat_VacuumStats.
Count visibility-map updates for empty pages, scanned pages and pages
revisited during heap cleanup, testing the all-visible bit when needed.

Collector delivery and SQL exposure follow in the reporting commit.

Include these per-operation measurements in VACUUM VERBOSE output.

Related to vm_new_visible_pages in v40-0006, "Extended vacuum statistics:
visibility-map page transitions for tables". That patch reports existing
PostgreSQL VM transition counters; here the all-visible measurement is
added to Cloudberry's PostgreSQL 14 vacuum paths. The patch's separate
all-frozen and combined visible/frozen counters are not included here.

Related-to: https://www.postgresql.org/message-id/attachment/199749/v40-0006-Extended-vacuum-statistics-visibility-map-page-trans.patch
@Alena0704
Count vacuum runs made aggressive by the freeze age
35bcc24 
Introduce freeze_age_vacuum_count and preserve the freeze-age
decision before DISABLE_PAGE_SKIPPING can make the scan aggressive.
Record one for a run driven by relfrozenxid or relminmxid, and zero
when aggressive scanning is forced only by DISABLE_PAGE_SKIPPING.
Include VACUUM FREEZE, which sets the freeze-age thresholds to zero.

This introduces the measurement; collector accumulation and SQL
exposure follow in the reporting commit.

Include these per-operation measurements in VACUUM VERBOSE output.
@Alena0704
Measure append-optimized vacuum phases and remaining dead tuples
01eea8b 
Preserve remaining dead tuples from post-cleanup and accumulate elapsed
and cost-delay time across AO vacuum phases. Measure index cleanup time
as well. Widen existing compaction tuple and byte accumulators to int64,
and keep the reclaimed-block conversion 64-bit as well.

Collector reporting and SQL-level AO tests follow in a separate commit.

Include these per-operation measurements in VACUUM VERBOSE output.

Measure cleanup-only index scans when there are no obsolete AO segments.
@michaelpq @Alena0704
Track per-relation cumulative time spent in [auto]vacuum and [auto]an…
acfdb4f 
…alyze

This commit adds four fields to the statistics of relations, aggregating
the amount of time spent for each operation on a relation:
- total_vacuum_time, for manual vacuum.
- total_autovacuum_time, for vacuum done by the autovacuum daemon.
- total_analyze_time, for manual analyze.
- total_autoanalyze_time, for analyze done by the autovacuum daemon.

This gives users the option to derive the average time spent for these
operations with the help of the related "count" fields.

Bump PGSTAT_FILE_FORMAT_ID for the additions in PgStat_StatTabEntry.
SQL access follows in the separate vacuum_stats extension commit; the
system catalog is unchanged.

Author: Sami Imseih
Reviewed-by: Bertrand Drouvot, Michael Paquier
Discussion: https://postgr.es/m/CAA5RZ0uVOGBYmPEeGF2d1B_67tgNjKx_bKDuL+oUftuoz+=Y1g@mail.gmail.com

(cherry picked from commit 30a6ed0)

(cherry picked from commit 6a99f9d)
(cherry picked from commit 5f76c5e)

Adaptation for this branch: carry elapsed time in UDP VACUUM/ANALYZE
messages and initialize the new counters in both relation-entry creation
paths. Keep microsecond precision internally and expose milliseconds.
Time AO vacuum from the first phase executed by this worker and pass that recorded
start time when reporting the final phase.
@Alena0704
Report per-index removed tuples in vacuum instrumentation
435d53e 
The per-index autovacuum log line reports page counts but not the number
of index entries removed. Add tuples_removed accumulated by the index's
bulkdelete passes to that line:

  index "t_pkey": tuples: 500 removed; pages: 30 in total, ...

Port v44-0001 to Cloudberry. On this PG14 base the summary is used by
autovacuum; manual VACUUM VERBOSE already reports removed index row
versions from lazy_cleanup_one_index(). Preserve that existing output.
This uses the access method's existing counter and adds no new statistics
collection or dependency on track_vacuum_statistics.

Suggested by Bharath Rupireddy (@BRupireddy2).

Based-on: https://www.postgresql.org/message-id/attachment/204702/v44-0001-Report-per-index-removed-tuples-in-vacuum-instru.patch
@Alena0704 @danolivo @zubkov-andrei
Track index and database vacuum time and cost-based delay
d192250 
Add cumulative elapsed and delay time for indexes and databases, and
delay time for tables. Keep manual VACUUM and autovacuum totals separate.
Database totals use table timings, which already include index work.

Adapt v44-0002 to Cloudberry's PostgreSQL 14 UDP collector, including AO
vacuum. Store times in microseconds and report delay in VACUUM VERBOSE
and autovacuum logs. Collection follows track_counts; delay measurement
also requires track_cost_delay_timing and excludes ANALYZE calls.

SQL access is provided separately through vacuum_stats. Bump the
statistics-file format for the new fields; the system catalog is unchanged.

Based-on: https://www.postgresql.org/message-id/attachment/204703/v44-0002-Track-vacuum-times-for-indexes-and-databases-and.patch
(cherry picked from commit 77a3ffa32d07d8a2947bf7e8dbb8d342ce05549d)

Co-authored-by: Andrei Lepikhov <lepihov@gmail.com>
Co-authored-by: Andrei Zubkov <zubkov@moonset.ru>
@Alena0704
Alena0704 force-pushed the vacuum_instrumentation_rel2 branch from ccadeff to d192250 Compare October 7, 2026 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@leborchuk leborchuk Awaiting requested review from leborchuk

At least 2 approving reviews are required to merge this pull request.

Assignees

No one assigned

Labels

type: Enhancement New feature or request, ideas

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Alena0704 @nathan-bossart @michaelpq