Skip to content

docs(dsql): add limits/error-code reference, fix wrong messages - #279

Open
praba2210 wants to merge 1 commit into
mainfrom
fix/dsql-limits-and-error-messages
Open

praba2210 wants to merge 1 commit into
mainfrom
fix/dsql-limits-and-error-messages

Conversation

@praba2210

@praba2210 praba2210 commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

references/troubleshooting.md is where an agent looks up a DSQL error before acting on it. Five of its quoted messages are not what DSQL emits — one matches nothing at all — so the lookup silently fails. Coverage also stopped at 40001 and 23503, leaving common failures like transaction row limit exceeded with no entry.

Related

Follows #278, which touched index-conversion.md. That PR said dsql_lint preserves index key sort order; this one reverses it — DSQL rejects sort direction outright. Read ## Index Key Sort Order as a correction to #278, not an addition.

Changes

  • New: ## Limits and Error Codes in troubleshooting.md — 22 limits, 20 rejections and 10 cluster quotas, each with its SQLSTATE and exact message. 54000 covers 15 of the rows and 0A000 covers 13, so the message is the discriminator, not the code.
  • Five wrong messages corrected: Token has expired, Transaction exceeds 3000 rows, Datatype array not supported, Please use CREATE INDEX ASYNC, OC001 - Concurrent DDL operation.
  • Four corrections change the advice: OC001 is 40001 and already inside the retry loop; unable to accept connection is a prefix shared by five conditions and only the Signature expired hint means a stale token; dsql_lint strips sort order rather than preserving it, and silently for ASC; an EOF at or after COMMIT leaves the outcome unknown, so a bare retry can double-write.
  • Limits corrected: key size is ~1,981 usable bytes shared across every key column and any secondary index; the 24-index limit counts the primary key, leaving 23 secondary. Two index examples drop DESC, which returns 0A000 today.
  • Evals: five added (sort order, transaction size, EOF-at-COMMIT, catalog change, key size). The grader accepted 24 secondary indexes and a fixture pinned that as passing, so the suite graded this PR's bug as a correct answer. It now rejects that framing worded either way, and no longer fails correct answers laid out as a table, bullet list or bolded number.

Two behaviours no AWS page documents, both measured here:

  • Secondary indexes consume the 10 MiB transaction budget, ~430 bytes per entry — accounted ≈ raw + rows × (~150 B + ~430 B per secondary index). A 3,000-row batch on a four-index table loses ~5 MiB before any payload counts, so a statement that commits against a bare table fails against an indexed one. The docs scope the 3,000-row limit as applying "regardless of the number of secondary indexes"; that independence does not carry to the size limit.
  • OC001 is any catalog change. GRANT and REVOKE raise it, one change is enough, it need not be concurrent, and only transactions that write are affected.

Verification

  • Every quoted message reproduced verbatim against a live cluster; plan shapes confirmed by EXPLAIN. Cluster quotas are documented only, not exercised.
  • 10 MiB figures measured with the payload in an unindexed column, so the key-size limit could not bind first and be mistaken for the size limit.
  • mise run build passes — 133 unit tests, 27 functional evals, 118 assertions.

Version 1.9.1 → 1.10.0 across the three manifests — minor: new guidance plus an eval change.

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.

Copilot AI balanced review requested due to automatic review settings September 30, 2026 22:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@praba2210
praba2210 force-pushed the fix/dsql-limits-and-error-messages branch from d43958d to a136fb4 Compare September 30, 2026 23:41
Copilot AI balanced review requested due to automatic review settings September 30, 2026 23:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI balanced review requested due to automatic review settings October 1, 2026 19:13
@praba2210
praba2210 force-pushed the fix/dsql-limits-and-error-messages branch from a136fb4 to 703c080 Compare October 1, 2026 19:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@praba2210
praba2210 force-pushed the fix/dsql-limits-and-error-messages branch 2 times, most recently from 10a0809 to 924d8e8 Compare October 1, 2026 21:22
Copilot AI balanced review requested due to automatic review settings October 1, 2026 21:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI balanced review requested due to automatic review settings October 1, 2026 22:29
@praba2210
praba2210 force-pushed the fix/dsql-limits-and-error-messages branch from 924d8e8 to 2706eca Compare October 1, 2026 22:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@praba2210
praba2210 marked this pull request as ready for review October 5, 2026 17:29
@praba2210
praba2210 requested review from a team as code owners October 5, 2026 17:29
Copilot AI balanced review requested due to automatic review settings October 5, 2026 18:12
@praba2210
praba2210 force-pushed the fix/dsql-limits-and-error-messages branch from 2706eca to d482636 Compare October 5, 2026 18:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@praba2210
praba2210 force-pushed the fix/dsql-limits-and-error-messages branch from d482636 to e62a465 Compare October 5, 2026 21:33
Copilot AI balanced review requested due to automatic review settings October 5, 2026 21:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@praba2210

praba2210 commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor Author

Self-review against a live cluster. 17 issues found and fixed.

# Issue Fix Verified
1 Grader failed correct answers laid out as a table, bullet list or bolded number — the shapes eval 3's own doc-call induces Match the number and index noun in either order; accept per table/can have/accepts as bounds Probe 12/14 wrong → 0/14
2 Grader passed the off-by-one it exists to catch, in 6 phrasings Added not including, doesn't count, plural primary keys, em dash, fronted clause, cross-sentence; scoped the inclusive exemption to the vetoing statement Same probe; 6 tests
3 "Counts as a warning" false for ASC Added the DESC/ASC severity table; gate on the diagnostic list, not counters ASC → {errors:0, warnings:0}; cluster rejects it
4 NULLS placement wrongly listed as auto-fixed by index_sort_direction Split NULLS out of that rule Lint: 0 diagnostics, retained; cluster accepts
5 "Any uniformly-ordered ORDER BY is served by a scan" — false Rewrote as uniform over a leading prefix; added the covering-projection precondition ORDER BY created_at → Sort; all 4 plan rows reproduced
6 10 MiB "not monotonic, neither predicts the outcome", plus a write-performing MUST Replaced with accounted ≈ raw + rows × (~150 B + ~430 B/index); MUST removed Linear across 0/1/2/4 indexes; predicts 200–2,900 rows to 1%
7 OC001 attributed to "concurrent DDL" — wrong in both words Redefined as any catalog change incl. GRANT/REVOKE, concurrency not required; AWS doc quoted 5/5 scenarios incl. GRANT, REVOKE, already-completed DDL
8 SKILL.md cited 10 MiB and routed to a file with zero mentions of it Routes to troubleshooting.md for behaviour, batched-migration.md for the pattern grep -c → 0
9 View size ~128 KiB labelled "observed" but is a derived ceiling, and not safe Relabelled to the 131–524 KiB observed range; ceiling → ~120 KiB, size by trial ~127 KiB OR-chain rejected
10 authentication-guide.md still said "regenerate token on connection errors" Scoped to the Signature expired hint Reproduced live
11 24-includes-the-PK asserted as if documented Marked as observed; quotas page states only the 24 PK table 23 max, no-PK table 24 max
12 HINT treated as part of the message line Named the field per driver; (OC001) matched as a substring Live psycopg error object
13 index_sort_direction absent from dsql-lint.md Added to the rule examples, plus a don't-gate-on-one-counter note grep -c → 0
14 Key budget stated as ~1,474 bytes Corrected to ~1,455, shared ceiling ~1,955, docs divergence flagged PK 10→1,946 / 500→1,455 / 1,000→955
15 MUST reconnect-and-confirm with no path when the outcome is unknowable Added MUST NOT retry / MUST report and stop —
16 No eval coverage for ~250 lines of new guidance 5 evals added: sort order, transaction size, EOF-at-COMMIT, catalog change, key size 131 tests pass
— Minors schema.md now qualifies the 24; negated heading → NULLS placement is retained; token-minting MUST NOT → affirmative; added the documented-vs-observed maximum column size exceeded row —

Two things no AWS page documents, both measured here:

  • Secondary indexes consume the 10 MiB transaction budget, ~429 bytes per entry. The docs explicitly scope the 3,000-row limit as applying "regardless of the number of secondary indexes"; that independence does not carry to the size limit. A 3,000-row batch on a four-index table loses ~5 MiB before any payload counts.
  • The 24-index limit includes the primary key. The quotas page says only "24".

mise run build passes, 131 tests.

troubleshooting.md covered only 40001 and 23503, and five of its
quoted error messages did not match what DSQL emits. One matches
nothing in the real text, so an agent looking it up finds nothing.

Adds a Limits and Error Codes section: 22 limits, 20 rejections and
10 cluster quotas, each with its SQLSTATE and exact message. 54000
covers 15 limit rows and 0A000 covers 13 rejections, so within the
section the message is the discriminator, not the code.

Four corrections change the advice:

- OC001 is 40001, already in the OCC retry loop. OC000 is the DML
  conflict. OC001 fires when a transaction's snapshot predates a
  catalog change, which includes GRANT and REVOKE as well as DDL,
  and whether or not the change is still in flight. A retry takes a
  fresh snapshot and clears a one-shot change; only a session that
  keeps changing the catalog needs serializing.
- "unable to accept connection" is a prefix shared by five
  conditions. Only the Signature expired hint means the token is
  stale, and regenerating cannot fix a skewed host clock. HINT is a
  separate protocol field, so the guidance names it per driver.
- An EOF at or after COMMIT leaves the commit outcome unknown, so a
  bare retry can double-write. Adds what to do when the outcome
  cannot be established at all.
- dsql_lint strips index sort order rather than preserving it, and
  the severity depends on the direction: DESC is a warning, ASC is
  neither an error nor a warning, so gating on either counter misses
  the rewrite. Gate on the diagnostic list instead.

Corrects key size to about 1,981 usable bytes, shared across all key
columns and with any secondary index, and the 24-index limit to
include the primary key, so a table with one accepts 23 secondary.
Both confirmed against a cluster: a 500-byte primary key leaves
1,455 bytes for a secondary index key, and totals hold near 1,955
whatever the split. Key size, temp space and view-definition size
all differ from the quotas page; each row now says so.

Secondary indexes also count against the 10 MiB transaction limit,
which is documented nowhere. Accounted size runs about 150 bytes per
row above raw, plus about 430 per secondary index entry, measured
linear across 0, 1, 2 and 4 indexes and predictive to within 1% from
200 to 2,900 rows. A 3,000-row batch on a four-index table loses
roughly 5 MiB before any payload counts.

The eval grader accepted "24 secondary indexes" and a fixture pinned
it as passing, so the suite graded the error this commit fixes as a
correct answer. It also failed correct answers laid out as a quoted
table, bullet list or bolded number, which is what an eval requiring
a documentation call tends to produce. The grader now matches the
number and the index noun in either order, accepts bare "per table"
and "can have" as bounds, and scopes the inclusive exemption to the
statement that triggered the veto so one correct sentence no longer
licenses a wrong claim elsewhere. Adds the excluding phrasings it
missed: not including, doesn't count, plural primary keys, a fronted
clause and an em dash. Six tests pin 20 phrasings.

Adds five evals for the sort-order, transaction-size, EOF-at-COMMIT,
catalog-change and key-size guidance, which had none.

Also drops DESC from two index examples that return 0A000 today. A
direction-free index serves a uniformly ordered ORDER BY by backward
scan when the keys are a leading prefix and the projection is
covered; mixed ordering adds a sort.

Bumps the plugin version to 1.10.0.
@praba2210
praba2210 force-pushed the fix/dsql-limits-and-error-messages branch from e62a465 to ebbd8a2 Compare October 5, 2026 23:11
Copilot AI balanced review requested due to automatic review settings October 5, 2026 23:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@praba2210

Copy link
Copy Markdown
Contributor Author

Second review pass against a live cluster.

Issue Fix
troubleshooting.md still defined OC001 as "Multiple DDL operations on the same resource" / "Concurrent DDL", contradicting eval 26 and occ-retry-patterns.md in this same PR Snapshot predates any catalog change; one statement is enough and need not be concurrent. A single already-committed ALTER TABLE, GRANT and REVOKE each raise it alone
Nothing stated that OC001 only affects transactions that write — which also made eval 26's read-only reporting session an impossible premise Added to both files; 4/4 read-only variants commit clean, including explicit BEGIN READ ONLY. Eval 26's job now writes
The grader still failed correct answers in two shapes: the singular the 24-index limit, and an answer that also stated truthfully that the primary key does not count toward the 1,600-column limit — the latter reported as the off-by-one it polices Singular accepted; veto now ignores statements about another limit's unit. Test for each
Overturned by the cluster, left as written OC001 is reported at COMMIT to the later committer; DSQL classifies GRANT/REVOKE as DDL itself (0A000: multiple ddl statements not supported in a transaction for a pair)
Out of scope, filing separately GRANT counts against the one-DDL-per-transaction rule, so references/access-control.md showing eight consecutive GRANT statements as one block fails

Also corrected the eval README's prompt and assertion counts and two stale wordings.

mise run build passes — 133 tests, 27 evals, 118 assertions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants