Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@
"serverless",
"postgresql"
],
"version": "1.8.1"
"version": "1.9.0"
},
{
"category": "deployment",
Expand Down
2 changes: 1 addition & 1 deletion plugins/databases-on-aws/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,5 @@
"license": "Apache-2.0",
"name": "databases-on-aws",
"repository": "https://github.com/awslabs/agent-plugins",
"version": "1.8.1"
"version": "1.9.0"
}
2 changes: 1 addition & 1 deletion plugins/databases-on-aws/.codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "databases-on-aws",
"version": "1.8.1",
"version": "1.9.0",
"description": "Expert database guidance for the AWS database portfolio. Design schemas, execute queries, handle migrations, and choose the right database for your workload.",
"author": {
"name": "Amazon Web Services",
Expand Down
36 changes: 18 additions & 18 deletions plugins/databases-on-aws/skills/dsql/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,9 @@ Load these files as needed for detailed guidance:

### ORM Guides:

| Reference | When to Load | Contains |
| ----------------------------------------------------------- | ------------------------- | ------------------------------------------------------------------------ |
| [orm-guides/overview.md](references/orm-guides/overview.md) | Migrating any ORM to DSQL | Adapter names, key gotchas for Django/EF Core/Hibernate/Rails/SQLAlchemy |
| Reference | When to Load | Contains |
| ----------------------------------------------------------- | ----------------------------------------------- | ------------------------------------------------------------------------------------- |
| [orm-guides/overview.md](references/orm-guides/overview.md) | Migrating an ORM or using ORM locking with DSQL | Adapter names, locking, and key gotchas for Django/EF Core/Hibernate/Rails/SQLAlchemy |

### Data Loading:

Expand Down Expand Up @@ -145,18 +145,18 @@ See [mcp-tools.md](mcp/mcp-tools.md) for detailed usage and examples.
Consult for verifying DSQL service limits before advising users. The numeric limits below are
defaults that may change — when a user's decision depends on an exact limit, verify it first:

| Limit | Default | Verify query |
| ------------------------------ | ------------- | ---------------------------------- |
| Max rows per transaction | 3,000 | `aurora dsql transaction limits` |
| Max data size per transaction | 10 MiB | `aurora dsql transaction limits` |
| Max transaction duration | 5 minutes | `aurora dsql transaction limits` |
| Max connections per cluster | 10,000 | `aurora dsql connection limits` |
| Auth token expiry | 15 minutes | `aurora dsql authentication token` |
| Max connection duration | 60 minutes | `aurora dsql connection limits` |
| Max indexes per table | 24 | `aurora dsql index limits` |
| Max columns per index | 8 | `aurora dsql index limits` |
| IDENTITY/SEQUENCE CACHE values | 1 or >= 65536 | `aurora dsql sequence cache` |
| Supported column data types | See docs | `aurora dsql supported data types` |
| Limit | Default | Verify query |
| ------------------------------------- | ------------- | ---------------------------------- |
| Max row modifications per transaction | 3,000 | `aurora dsql transaction limits` |
| Max data size per transaction | 10 MiB | `aurora dsql transaction limits` |
| Max transaction duration | 5 minutes | `aurora dsql transaction limits` |
| Max connections per cluster | 10,000 | `aurora dsql connection limits` |
| Auth token expiry | 15 minutes | `aurora dsql authentication token` |
| Max connection duration | 60 minutes | `aurora dsql connection limits` |
| Max indexes per table | 24 | `aurora dsql index limits` |
| Max columns per index | 8 | `aurora dsql index limits` |
| IDENTITY/SEQUENCE CACHE values | 1 or >= 65536 | `aurora dsql sequence cache` |
| Supported column data types | See docs | `aurora dsql supported data types` |

**When to verify:** Before recommending batch sizes, connection pool settings, or schema designs where hitting a limit would cause failures; any time the exact number can affect user decision.

Expand Down Expand Up @@ -271,9 +271,9 @@ MUST load [query-plan/workflow.md](references/query-plan/workflow.md) at entry

MUST load [pg-migrations/type-mapping.md](references/pg-migrations/type-mapping.md), [pg-migrations/schema-objects.md](references/pg-migrations/schema-objects.md), and [foreign-keys.md](references/foreign-keys.md). Run `dsql_lint(fix=true)` first for mechanical fixes, preserve foreign-key relationships, translate unsupported source syntax or options, then apply semantic conversions from the pg-migrations references for unfixable diagnostics and patterns the linter cannot handle. Re-lint the final output before deploying.

### Workflow 11: ORM Migration (Django/EF Core/Hibernate/Rails/SQLAlchemy)
### Workflow 11: ORM Migration and Locking (Django/EF Core/Hibernate/Rails/SQLAlchemy)

Load [orm-guides/overview.md](references/orm-guides/overview.md) for adapter names and framework-specific gotchas.
Load [orm-guides/overview.md](references/orm-guides/overview.md) for adapter names, locking guidance, and framework-specific gotchas.

### Workflow 12: System Diagnostics (CloudWatch AAS)

Expand All @@ -290,7 +290,7 @@ MUST load [system-diagnostics/workflow.md](references/system-diagnostics/workflo
- **OCC serialization error:** Retry the transaction. If persistent, check for hot-key contention — see [troubleshooting.md](references/troubleshooting.md).
- **Foreign key violation (`23503`):** Correct the relationship or referential action; **MUST NOT**
send it through the `40001` retry loop — see [troubleshooting.md](references/troubleshooting.md).
- **Transaction exceeds limits:** Split into batches under 3,000 rows — see [batched-migration.md](references/ddl-migrations/batched-migration.md).
- **Transaction exceeds limits:** Split write batches to stay under 3,000 row modifications and 10 MiB; locked-row primary keys count toward the size limit — see [batched-migration.md](references/ddl-migrations/batched-migration.md).
- **Token expiration mid-operation:** Generate a fresh IAM token — see [authentication-guide.md](references/auth/authentication-guide.md). See [troubleshooting.md](references/troubleshooting.md) for other issues.

## Additional Resources
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ transact([
**Important Notes:**

- Each ALTER TABLE must be in its own transaction (DSQL limitation)
- Keep transactions under 3,000 rows and 10 MiB
- Keep transactions under 3,000 row modifications and 10 MiB
- For large batch operations, split into multiple transact calls
- **MUST** build every statement with [`safe_query.build()`](safe_query.py).
Write mode disables all server-side injection filters
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ Aurora DSQL is designed for massive horizontal scale without latency degradation

### Batch Size Optimization

- **PREFER batches of 500-1,000 rows** - Balance throughput and transaction limits (3,000 rows, 10 MiB, 5 minutes max — verify via `awsknowledge`: `aurora dsql transaction limits`)
- **PREFER batches of 500-1,000 rows** - Balance throughput and transaction limits (3,000 row modifications, 10 MiB, 5 minutes max — verify via `awsknowledge`: `aurora dsql transaction limits`)
- **SHOULD process batches concurrently** - Use multiple connections; consider multiple threads for bulk loading
- **Smaller batches reduce** lock contention, enable better concurrency, fail faster, distribute load evenly
- **Smaller batches reduce** OCC conflict surface, enable better concurrency, fail faster, distribute load evenly

### AVOID Hot Keys

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ For the full Table Recreation Pattern and verify & swap steps, see [overview.md]
## Batch Size Rules

- **PREFER batches of 500-1,000 rows** for optimal performance
- Smaller batches reduce lock contention and enable better concurrency
- Smaller batches reduce OCC conflict surface and enable better concurrency

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ PRIMARY KEY, UNIQUE, FOREIGN KEY, NOT NULL, CHECK, DEFAULT (CREATE TABLE or dire
Verify current limits via `awsknowledge`: `aurora dsql transaction limits`

```
Rows: 3,000 max
Row modifications: 3,000 max
Size: 10 MiB max
Duration: 5 minutes max
Isolation: Repeatable Read (fixed)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ DELETE FROM owner WHERE city = 'Portland';

**Transaction Limits** (verify current limits via `awsknowledge`: `aurora dsql transaction limits`)**:**

- Maximum 3,000 rows per transaction
- Maximum 3,000 row modifications per transaction
- Maximum 10 MiB data size per transaction
- Maximum 5 minutes per transaction

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,18 +26,32 @@ Django and Rails; EF Core, Hibernate, and SQLAlchemy provide composite relations
| Rails | Standard `pg` gem + `aws-sdk-dsql` | `gem 'pg'` + `gem 'aws-sdk-dsql'` |
| SQLAlchemy | `aurora_dsql_sqlalchemy` | `pip install aurora-dsql-sqlalchemy boto3` |

## SELECT FOR UPDATE

Aurora DSQL does not require equality predicates on every primary-key column for
`SELECT ... FOR UPDATE`; non-key predicates and queries that join multiple tables are supported.
It does not take blocking row locks. Instead, rows targeted by the locking clause participate in
optimistic commit-time conflict checks. If a concurrent transaction changes a targeted row, the
transaction that loses the commit race fails with SQLSTATE `40001`; retry the whole transaction
with backoff. Keep external side effects outside the retried callback or make them idempotent.

The primary key of each row targeted by the locking clause counts toward the 10 MiB
transaction-size limit.

`FOR UPDATE` and `FOR KEY SHARE` are supported. `FOR NO KEY UPDATE` and `FOR SHARE` are not supported.

## Key Gotchas Per Framework

### Django

| Issue | Fix |
| ----------------- | ------------------------------------------------------------------------------- |
| ENGINE | `'aurora_dsql_django'` (not `django.db.backends.postgresql`) |
| CONN_MAX_AGE | ≤ 1800 (DSQL timeout is 1 hour) |
| Migrations | Each DDL in its own migration; `RunSQL("CREATE INDEX ASYNC ...")` |
| SELECT FOR UPDATE | Use when a write depends on rows read; retain whole-transaction OCC retry |
| AutoField | Replace with `UUIDField(primary_key=True, default=uuid.uuid4)` |
| ForeignKey | Keep `ForeignKey`; the DSQL backend creates database constraints for new tables |
| Issue | Fix |
| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| ENGINE | `'aurora_dsql_django'` (not `django.db.backends.postgresql`) |
| CONN_MAX_AGE | ≤ 1800 (DSQL timeout is 1 hour) |
| Migrations | Each DDL in its own migration; `RunSQL("CREATE INDEX ASYNC ...")` |
| SELECT FOR UPDATE | `select_for_update()` supports non-key filters and joined querysets that use inner joins; retry the whole transaction on SQLSTATE `40001` |
| AutoField | Replace with `UUIDField(primary_key=True, default=uuid.uuid4)` |
| ForeignKey | Keep `ForeignKey`; the DSQL backend creates database constraints for new tables |

### EF Core (.NET)

Expand All @@ -60,19 +74,20 @@ Requires .NET 8.0+, EF Core 9.0.7+, and `Amazon.AuroraDsql.Npgsql` 1.1.0+.
| Dialect | Provided by `aurora-dsql-hibernate-dialect` (auto-registered) |
| ID generation | `@GeneratedValue(strategy = GenerationType.UUID)` |
| OCC retry | Prefer the [aurora-dsql-jdbc-connector](https://github.com/awslabs/aurora-dsql-connectors/tree/main/java/jdbc) — built-in retry for SQLSTATE 40001. For manual `@Retryable`, match on `SQLException` and check `getSQLState() == "40001"` (Hibernate's class-40 mapping varies by version). |
| Locking | `PESSIMISTIC_WRITE` supports non-key predicates and inner-joined queries; it adds commit-time OCC checks rather than blocking row locks |
| FK constraints | Keep normal relationship mappings; the DSQL dialect exports foreign key constraints |
| DDL generation | `hibernate.hbm2ddl.auto = none` — manage DDL manually |

### Rails

| Issue | Fix |
| ---------- | ------------------------------------------------------------------------------------------------------------------- |
| adapter | `postgresql` (standard pg gem) |
| Auth | Custom connection handler generating IAM tokens via `aws-sdk-dsql` |
| Migrations | `disable_ddl_transaction!` in each migration |
| PKs | `id: :uuid` in `create_table` |
| FKs | Use `add_foreign_key ..., validate: false`, then run `ALTER TABLE ASYNC ... VALIDATE CONSTRAINT` and verify the job |
| Locking | Use `lock!` / `with_lock` when a decision depends on rows read; retain OCC retry in `ApplicationRecord` |
| Issue | Fix |
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| adapter | `postgresql` (standard pg gem) |
| Auth | Custom connection handler generating IAM tokens via `aws-sdk-dsql` |
| Migrations | `disable_ddl_transaction!` in each migration |
| PKs | `id: :uuid` in `create_table` |
| FKs | Use `add_foreign_key ..., validate: false`, then run `ALTER TABLE ASYNC ... VALIDATE CONSTRAINT` and verify the job |
| Locking | Use `Relation#lock` for non-key filters and joins inside a transaction; use `lock!` or `with_lock` for individual persisted records; retry the whole transaction on SQLSTATE `40001` |

### SQLAlchemy

Expand Down
Loading
Loading