Skip to content

feat: seed UI collection watches from the list resourceVersion#6311

Merged
rpelczar merged 6 commits into
Marvin9/project-grpc-rest-migrationfrom
jacobboykin/resource-version-watch
Jun 22, 2026
Merged

feat: seed UI collection watches from the list resourceVersion#6311
rpelczar merged 6 commits into
Marvin9/project-grpc-rest-migrationfrom
jacobboykin/resource-version-watch

Conversation

@jacobboykin

@jacobboykin jacobboykin commented May 18, 2026

Copy link
Copy Markdown
Contributor

What this does

On big projects the pipeline page can feel loaded-but-frozen: the content paints, and then for several seconds clicks and menus do nothing. The culprit is the collection watches. When the UI lists a collection — Stages, Warehouses, Promotions, Freight — and then opens its live watch, the watch starts from scratch and the API server replays every object it just listed back as an individual ADDED event. On a large project that's thousands of redundant events the browser has to decode and push through React one at a time, and that's what blocks the main thread.

This wires up the standard Kubernetes list-then-watch pattern so it doesn't happen. The list response carries its resourceVersion and the follow-up watch starts from it instead of replaying everything, so the watch opens and stays quiet.

How it works

  • The list and query endpoints return the list-level resourceVersion. Stages, Warehouses, and Promotions already carry one; QueryFreight now returns one too.
  • The watch endpoints accept a resourceVersion and start the Kubernetes watch from there.
  • On the front end, each watch seeds itself from its list query's resourceVersion. The seed is read once when the stream opens and kept out of the effect deps on purpose, so an incoming event updating the cache doesn't tear the stream down and immediately reopen it.
  • If the seed is too old to watch from, the server says so and the client refetches the list for a fresh resourceVersion and reopens, instead of spinning on an expired one.
  • Filters that have to be applied in-process (warehouse, stage, origin) emit a synthetic delete when an object drops out of the filter, so the client still sees it leave.
  • Seed lists read through a dedicated uncached client so the resourceVersion handed back is actually fresh and watchable — the cached client can return one that's already too old and send you straight into a retry loop. Those direct reads stay scoped to the seed path and are authorized exactly like the normal client first, so nothing sidesteps Kargo's permissions.

Impact

On a project with roughly a thousand Stages and Freight, opening the pipeline page used to replay the whole collection — about 960 Stage and 1,300 Freight events on a single open. With seeding the watches replay nothing. The Freight stream alone drops from around 1,300 events and a megabyte and a half of data down to zero.

What matters is the effect on the page: main-thread blocking during load falls by roughly 85%, and the stretch where the UI is unresponsive shrinks from over ten seconds to a few. It's an interaction-responsiveness win rather than a faster first paint — the initial list still has to download and render — and it only shows up at scale. Small projects won't notice it; the large ones that motivated this go from frozen for the better part of twenty seconds to actually usable.

@netlify

netlify Bot commented May 18, 2026

Copy link
Copy Markdown

Deploy Preview for docs-kargo-io ready!

Name Link
🔨 Latest commit b0bf5eb
🔍 Latest deploy log https://app.netlify.com/projects/docs-kargo-io/deploys/6a2c54c440021c0008c7f611
😎 Deploy Preview https://deploy-preview-6311.docs.kargo.io
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@kargo-governance-bot kargo-governance-bot Bot added needs/area Issue or PR needs to be labeled to indicate what parts of the code base are affected needs/kind Issue or PR needs to be labeled to clarify its nature needs/priority Priority has not yet been determined; a good signal that maintainers aren't fully committed labels May 18, 2026
@codecov

codecov Bot commented May 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 59.16667% with 98 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.52%. Comparing base (4b89b86) to head (238f9f6).

Files with missing lines Patch % Lines
pkg/server/watch_helpers.go 63.63% 20 Missing and 4 partials ⚠️
pkg/server/list_promotions_v1alpha1.go 31.03% 15 Missing and 5 partials ⚠️
pkg/server/kubernetes/client.go 36.66% 18 Missing and 1 partial ⚠️
pkg/server/query_freights_v1alpha1.go 55.00% 15 Missing and 3 partials ⚠️
pkg/server/list_stages_v1alpha1.go 78.37% 5 Missing and 3 partials ⚠️
pkg/server/resource_version.go 63.63% 3 Missing and 1 partial ⚠️
pkg/server/list_warehouses_v1alpha1.go 72.72% 2 Missing and 1 partial ⚠️
pkg/server/list_for_watch_seed.go 87.50% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@                           Coverage Diff                           @@
##           Marvin9/project-grpc-rest-migration    #6311      +/-   ##
=======================================================================
+ Coverage                                58.49%   58.52%   +0.03%     
=======================================================================
  Files                                      501      503       +2     
  Lines                                    42226    42390     +164     
=======================================================================
+ Hits                                     24700    24809     +109     
- Misses                                   16030    16075      +45     
- Partials                                  1496     1506      +10     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jacobboykin

This comment was marked as outdated.

@jacobboykin jacobboykin reopened this Jun 12, 2026
@jacobboykin jacobboykin force-pushed the jacobboykin/resource-version-watch branch from c667c73 to b0bf5eb Compare June 12, 2026 18:49
@jacobboykin jacobboykin changed the base branch from main to Marvin9/project-grpc-rest-migration June 12, 2026 19:42
@jacobboykin jacobboykin force-pushed the jacobboykin/resource-version-watch branch from b0bf5eb to 9518bc3 Compare June 12, 2026 19:48
@jacobboykin jacobboykin changed the title feat: add resourceVersion to Freight, Stage, Warehouse list and watch endpoints feat: seed UI collection watches from the list resourceVersion Jun 12, 2026
On large projects the UI's Stage/Warehouse/Promotion/Freight collection
streams open their follow-up watch with no resourceVersion, so the API
server replays every listed object as an individual ADDED event. On a big
project that is thousands of redundant events the browser decodes one at a
time, freezing the page for the better part of 20s after content appears.

Implement standard Kubernetes list-then-watch seeding on the REST/SSE path.

Server:
- Return the list-level resourceVersion from the list/query endpoints
  (Stages, Warehouses, Promotions natively; Freight via a new
  resourceVersion field on the QueryFreights response).
- Accept ?resourceVersion= on the SSE watch endpoints and start the watch
  from it via buildWatchListOptions.
- Read seed lists through an uncached direct reader (listForWatchSeed),
  gated by an explicit SubjectAccessReview that mirrors the cached client,
  falling back to the cached client when no direct reader is available.
- Surface an expired resourceVersion as an SSE error event so clients can
  relist, and emit synthetic DELETEs for client-side filtered watches.

UI:
- Centralize seeding in runSeededWatch: seed each watch from the matching
  list query's resourceVersion (kept out of effect deps to avoid reconnect
  churn), relist and reopen on an expired-resourceVersion error, and skip
  replayed ADDED events.

On a ~1k-object project this drops ~2,261 replayed ADDED events per page
load to 0 and reduces main-thread blocking during load by ~85%.

Signed-off-by: Jacob Boykin <jacob.boykin@akuity.io>
@jacobboykin jacobboykin force-pushed the jacobboykin/resource-version-watch branch from 9518bc3 to 899ee68 Compare June 12, 2026 20:28
The list+watch seed endpoints read through listForWatchSeed's uncached
directReader, which always returns a real list-level resourceVersion. The
effectiveResourceVersion max-item fallback only ran on the degraded
no-directReader path (tests, or no rest.Config), where its best-effort RV
is no better than opening the watch unseeded -- the documented pre-change
behavior. Replace it with a small normalizeListResourceVersion that maps
the cached client's "0" sentinel to an empty string, and collapse the
three near-duplicate resourceVersionFor*List helpers into inline calls.

The upsertOrDelete ADDED dedup guard is retained: it makes the degraded
unseeded path (which now replays) cheap.

Net -120 lines. No behavior change on the production directReader path.

Signed-off-by: Jacob Boykin <jacob.boykin@akuity.io>
Replace the bespoke uncached client built in the server package with the
cluster's existing GetAPIReader, exposed via kubernetes.Client.APIReader.
This is the same direct-read pattern the controllers and external webhooks
server use, and it drops the parallel client.New, its RESTMapper/connection,
and the RestConfig-into-server wiring. listForWatchSeed now reads through
client.APIReader and authorizes the caller first, single-mode.

Signed-off-by: Jacob Boykin <jacob.boykin@akuity.io>
…ion' into jacobboykin/resource-version-watch

Signed-off-by: Jacob Boykin <jacob.boykin@akuity.io>

# Conflicts:
#	ui/src/features/project/pipelines/use-watch-stages.ts
#	ui/src/features/project/pipelines/use-watch-warehouses.ts
#	ui/src/features/project/pipelines/watch-utils.ts
@jacobboykin jacobboykin marked this pull request as ready for review June 18, 2026 18:16
@jacobboykin jacobboykin requested a review from a team as a code owner June 18, 2026 18:16
…ranch 'origin' into jacobboykin/resource-version-watch
@Marvin9 Marvin9 requested a review from a team as a code owner June 19, 2026 10:37

@Marvin9 Marvin9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UI LGTM. Tested and checked the UI code.

@rpelczar rpelczar merged commit 030fb79 into Marvin9/project-grpc-rest-migration Jun 22, 2026
14 checks passed
@rpelczar rpelczar deleted the jacobboykin/resource-version-watch branch June 22, 2026 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs/area Issue or PR needs to be labeled to indicate what parts of the code base are affected needs/kind Issue or PR needs to be labeled to clarify its nature needs/priority Priority has not yet been determined; a good signal that maintainers aren't fully committed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants