Skip to content

[WIP] Add analyze-db.sh script for post-restore database warmup#3790

Draft
mstaeble wants to merge 2 commits into
openshift:mainfrom
mstaeble:worktree-analyze-db-script
Draft

[WIP] Add analyze-db.sh script for post-restore database warmup#3790
mstaeble wants to merge 2 commits into
openshift:mainfrom
mstaeble:worktree-analyze-db-script

Conversation

@mstaeble

@mstaeble mstaeble commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds scripts/analyze-db.sh to warm up the database after cloning/restoring the production database to staging
  • Runs three warmup steps via a detached pod:
    1. ANALYZE VERBOSE — updates planner statistics
    2. REINDEX DATABASE CONCURRENTLY — forces index pages off lazy-loaded EBS storage
    3. Cache warming — SELECT count(*) on all tables, scoped to the last 30 days for tables with date/timestamp columns, to pull heap pages off S3
  • Pod runs detached so your local machine does not need to stay connected; use --wait to block until completion
  • Adds argument validation to scripts/backfill-summaries.sh for missing flag values
  • Documents the new script in README.md

Test plan

  • Run ./scripts/analyze-db.sh --dry-run to verify dry-run output
  • Run ./scripts/analyze-db.sh against staging to verify ANALYZE and REINDEX complete
  • Run ./scripts/analyze-db.sh against staging to verify cache warming logs per-table row counts
  • Run ./scripts/backfill-summaries.sh --table (no value) to verify the new error message

@coderabbitai ignore

🤖 Generated with Claude Code

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci

openshift-ci Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 20, 2026
@mstaeble

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Full review finished.

@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@mstaeble, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 16 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 3d7c3a5d-a7ba-4aa1-945b-caaf34a7dbf9

📥 Commits

Reviewing files that changed from the base of the PR and between 52420bf and 3b8b69d.

📒 Files selected for processing (3)
  • README.md
  • scripts/analyze-db.sh
  • scripts/backfill-summaries.sh

Walkthrough

Adds a configurable ANALYZE VERBOSE script for restored PostgreSQL databases, documents its usage, and strengthens missing-value validation for options in the backfill summaries script.

Changes

Database analysis workflow

Layer / File(s) Summary
Database analysis script
scripts/analyze-db.sh, README.md
The new script parses namespace, database secret, and dry-run options; optionally creates a pod that runs ANALYZE VERBOSE using the configured database DSN. README documents defaults and overrides.

Backfill argument validation

Layer / File(s) Summary
Backfill option validation
scripts/backfill-summaries.sh
Value-taking options now reject missing arguments with an error instead of assigning an absent positional value.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Operator
  participant AnalyzeScript
  participant KubernetesPod
  participant PostgreSQL
  Operator->>AnalyzeScript: invoke analysis options
  AnalyzeScript->>KubernetesPod: delete and create analysis pod
  KubernetesPod->>PostgreSQL: run ANALYZE VERBOSE through psql
  PostgreSQL-->>KubernetesPod: return analysis results
  KubernetesPod-->>AnalyzeScript: complete pod command
Loading
🚥 Pre-merge checks | ✅ 20 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Test Coverage For New Features ⚠️ Warning New shell behavior in analyze-db.sh and backfill-summaries.sh has no automated tests; the PR only changes README and scripts. Add automated shell tests (e.g., bats/shunit) for analyze-db dry-run/args and backfill missing-flag-value errors.
✅ Passed checks (20 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Go Error Handling ✅ Passed This PR only changes README and shell scripts; no Go code or error-handling paths are touched, so the Go error-handling check is not applicable.
Sql Injection Prevention ✅ Passed No SQL is built from user input; analyze-db.sh runs fixed ANALYZE VERBOSE;, and backfill-summaries.sh only adds shell arg validation.
Excessive Css In React Should Use Styles ✅ Passed PR changes only README and shell scripts; no React components or inline CSS were added, so the style rule is not applicable.
Single Responsibility And Clear Naming ✅ Passed The PR only adds two focused scripts with descriptive names; each script handles one workflow and no broad packages/structs/methods were introduced.
Feature Documentation ✅ Passed No relevant docs/features page covers the new db ops scripts; the only feature doc is about job symptoms, and README was updated instead.
Stable And Deterministic Test Names ✅ Passed PR only changes README and shell scripts; no test files or Ginkgo titles were added or modified.
Test Structure And Quality ✅ Passed No Ginkgo test files were changed; the PR only updates README and shell scripts, so this check is not applicable.
Microshift Test Compatibility ✅ Passed Only README and shell scripts changed; no new Ginkgo e2e tests or MicroShift-unsafe APIs/features were added.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PR changes only README and shell scripts; no new Ginkgo e2e tests or SNO-sensitive assumptions were added.
Topology-Aware Scheduling Compatibility ✅ Passed Only docs and utility scripts changed; neither adds node selectors, affinity, spread rules, replica logic, or topology-specific scheduling assumptions.
Ote Binary Stdout Contract ✅ Passed PR only changes README and shell scripts; no process-level OTE binary code or stdout contract changes were introduced.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PR changes only README and shell scripts; no new Ginkgo e2e tests or IPv4-specific test code were added.
No-Weak-Crypto ✅ Passed The PR adds no weak/custom crypto or secret/token comparisons; touched files only add ANALYZE automation, flag validation, and docs.
Container-Privileges ✅ Passed Changed files add no privileged/host* settings, no SYS_ADMIN, and no explicit root or allowPrivilegeEscalation=true settings in the generated pod specs.
No-Sensitive-Data-In-Logs ✅ Passed No new log output exposes secrets, tokens, PII, or customer data; added messages are generic status text.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: adding analyze-db.sh for post-restore database warmup.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/analyze-db.sh`:
- Around line 40-54: Add the required top-level --image option to the oc run
invocation using the same PostgreSQL image already specified in the overrides,
and retain --rm so the temporary pod is removed automatically.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: f4078982-680e-4550-8e87-5535d4197238

📥 Commits

Reviewing files that changed from the base of the PR and between 6b1a47a and 52420bf.

📒 Files selected for processing (3)
  • README.md
  • scripts/analyze-db.sh
  • scripts/backfill-summaries.sh

Comment thread scripts/analyze-db.sh Outdated
@mstaeble
mstaeble force-pushed the worktree-analyze-db-script branch from 8399b4d to 2391d59 Compare July 20, 2026 22:22
@openshift-ci openshift-ci Bot added the ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review label Jul 20, 2026
@mstaeble
mstaeble marked this pull request as ready for review July 21, 2026 00:50
@mstaeble mstaeble changed the title [WIP] Add analyze-db.sh script for post-restore database warmup Add analyze-db.sh script for post-restore database warmup Jul 21, 2026
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 21, 2026
@openshift-ci
openshift-ci Bot requested review from dgoodwin and neisw July 21, 2026 00:50
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e

@mstaeble
mstaeble force-pushed the worktree-analyze-db-script branch from 2391d59 to 42b6154 Compare July 23, 2026 18:01
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e

After cloning or restoring the production database to staging, query
performance is degraded due to stale planner statistics and lazy-loaded
EBS storage. This script warms up the database via a detached pod that
runs ANALYZE VERBOSE and REINDEX DATABASE CONCURRENTLY.

Also adds argument validation to backfill-summaries.sh to produce clear
error messages when a flag value is missing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mstaeble
mstaeble force-pushed the worktree-analyze-db-script branch from 42b6154 to 12f9de7 Compare July 25, 2026 12:36
@openshift-ci

openshift-ci Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mstaeble

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 25, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e

Adds a third warmup step that runs SELECT count(*) on all user tables
to pull heap pages off lazy-loaded EBS storage. Tables with
date/timestamp columns are scoped to the last 30 days for partition
pruning. Partition children are skipped (queried through the parent).

The pod command is refactored from inline shell to a heredoc-based
script written to a temp file, avoiding deep escaping.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mstaeble

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e

@openshift-ci

openshift-ci Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

@mstaeble: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@mstaeble mstaeble changed the title Add analyze-db.sh script for post-restore database warmup [WIP] Add analyze-db.sh script for post-restore database warmup Jul 26, 2026
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 26, 2026
@mstaeble
mstaeble marked this pull request as draft July 26, 2026 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant