[olsonanl]

Summary

Implements the client half of the Data API's 429 Too Many Requests contract
(BV-BRC-API Docs/CLIENT-429.md).
The API currently ships with admission control disabled specifically because this
contract is not implemented in clients. This is what unblocks enabling it.

The governing rule is not "retry on 429" — it is back off by the amount the server asked
for, and give up rather than hammer. A client that retries a shed request badly makes the
overload worse at the moment the system has least capacity to absorb it; retry
amplification of exactly this kind caused the 2026-06-05 coordinator crash.

Ships behind enableThrottleRetry, default false. Merging changes nothing until a
deployment opts in — see Enabling below.

What's here

util/retryRequest.js	retry core — Retry-After, jitter, caps, 429/503 only
util/throttledRequest.js	dojo/request transport provider
widget/ThrottleBanner.js + css	user-visible busy state
store/P3JsonRest.js	grid-query coverage (separate transport)
p3-web.conf.sample	drive-by fix, see below

Per the spec: honours Retry-After and never substitutes a shorter delay; jitters
every wait by up to +30% so simultaneously-shed clients don't all return at the same
instant; caps interactive retries at 3; treats 503 identically and 504/400 as
non-retriable; floors every delay at 1s so Retry-After: 0 can't produce a tight loop.

An unattended mode (spec §3a — cap the delay at a 300s ceiling, keep going) is
implemented and tested but unused: the spec classifies the website as interactive. It
exists so downloads can use it later without a rewrite.

One registration covers ~200 call sites

dojo/request resolves through request/default, which honours
has('config-requestProvider') — set from the dojoConfig in views/javascript.ejs. 208
of the p3/ imports are dojo/request (vs 10 direct dojo/request/xhr), so a single
provider registration covers the widget layer with no call-site edits. Only Data API
URLs are matched; workspace/app-service/Shock/user-service traffic falls through to
./xhr untouched.

Two transports bypass it and are handled separately: P3JsonRest (legacy
dojo/_base/xhr) is covered here; BVBRCClient (fetch) is not — see Known gaps.

Why the banner is not optional

Testing the provider alone against injected 429s rendered a grid with 0 rows: the facet
and taxonomy queries retried and recovered, while the POST that fetches the actual rows was
shed and dropped. That is the single outcome the spec calls worst — "a user who sees zero
rows will believe their data is missing." Exhaustion renders as 1 - 1 of 0 results,
indistinguishable from a legitimate empty result.

So P3JsonRest coverage and the banner ship together with the provider. Any subset makes
throttling less visible, not more.

Verified live

Against a local server with injected 429s (Retry-After: 3):

429 -> 429 -> 200   gaps 3701, 4129 ms   POST genome/      (grid rows)
429 -> 429 -> 200   gaps 3554, 4552 ms   data/taxon_category/
grid rows rendered: 200

• banner: "Retrying in 5 seconds…" counting down, hidden on recovery; "The server is currently busy. Please try again in a few minutes." and persists when exhausted
• cancel mid-backoff: 0 further requests on the wire, counter balanced
• flag off, 429s in flight: 0 retries, 0 topics, no banner — identical to pre-feature
• no throttling: banner element never created, results.total 10349515 matching the
  pager, 0 page errors
• production build: verified the Dojo builder leaves has('config-requestProvider') as
  a runtime lookup while folding has('host-browser') to a literal, so the hook survives

75 tests pass (27 for retryRequest).

Two silent regressions caught

Both on the success path with no 429 involved — they'd have broken every grid in normal
operation:

1. results.total read the xhr off results.ioArgs, a property of the specific Deferred
   rawXhrPost returns. retryRequest hands back a different promise, so that
   back-channel vanishes.
2. dojo/Deferred freezes its promise (Deferred.js:305), so results.total = … is a
   silent no-op. Fixed by delegating first — what QueryResults itself does for the same
   reason.

Enabling

{ "enableThrottleRetry": true }

Gated in two places, not one: P3JsonRest calls retryRequest directly and never sees the
provider, so a flag covering only the registration would leave the grid retrying while its
facets did not — the half-enabled state that produces the empty grid above.

Off means absent, not inert: retryRequest returns the caller's own promise object
untouched, and the registry matcher itself tests the flag, so with the feature off every
request takes the stock ./xhr path with dojo's own promise rather than the rebuilt one.

What reviewers should kick

Retry behaviour is well covered. The open question is different: does routing all
dojo/request traffic through a rebuilt promise break anything that consumes a request
promise unusually?

Live testing covered /view/GenomeList/ genomes + features tabs only. Unexercised:
BLAST/Homology, Compare Regions, JBrowse, heatmaps, workspace browsing, job submission,
Copilot. A failure there would likely look like a silently empty panel rather than a console
error.

Two known deviations in the wrapped promise, both believed benign, neither exercised beyond
those two pages:

• not an instanceof dojo/promise/Promise — only when.js:36 checks, and its non-native
  branch is the correct path for a foreign thenable
• no .progress() — nothing in dgrid or dojo/store calls it, and XHR has no progress
  channel

then / otherwise / always / cancel / response and error .status are all present
and verified.

Known gaps (deliberate, documented in the plan)

• BVBRCClient (fetch) — still throws an unretried 429. Needs a browserify rebuild
  of the vendored bundle, and the status preserved at the throw site (it currently throws
  Error(statusText), discarding the code and headers, so this cannot be fixed from
  outside the client). Next PR.
• Downloads — form.submit() / window.open() hand the response to the browser, so no
  wrapper can reach them. They are also the most likely to be shed
  (DownloadTooltipDialog.js:106 requests limit(2500000)). Converting to XHR+Blob is the
  fix and the natural first unattended caller.

Drive-by: p3-web.conf.sample was not valid JSON

nconf parses p3-web.conf as JSON (config.js:62), so copying the sample — what the
README tells you to do — crashed the server at startup. Single-quoted keys have been there
since e0f8a55dd (2018-05-14), an autoformat pass that made it valid JavaScript and invalid
JSON. Happy to split this out if preferred.

🤖 Generated with Claude Code