Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@
"serverless",
"postgresql"
],
"version": "1.8.1"
"version": "1.9.0"
},
{
"category": "deployment",
Expand Down
2 changes: 1 addition & 1 deletion plugins/databases-on-aws/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,5 @@
"license": "Apache-2.0",
"name": "databases-on-aws",
"repository": "https://github.com/awslabs/agent-plugins",
"version": "1.8.1"
"version": "1.9.0"
}
2 changes: 1 addition & 1 deletion plugins/databases-on-aws/.codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "databases-on-aws",
"version": "1.8.1",
"version": "1.9.0",
"description": "Expert database guidance for the AWS database portfolio. Design schemas, execute queries, handle migrations, and choose the right database for your workload.",
"author": {
"name": "Amazon Web Services",
Expand Down
12 changes: 6 additions & 6 deletions plugins/databases-on-aws/skills/dsql/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
name: dsql
description: "Build with Aurora DSQL — manage schemas, execute queries, handle migrations, diagnose query plans, diagnose cluster performance, load data, and develop applications with a serverless, distributed SQL database. Covers IAM auth, multi-tenant patterns, MySQL-to-DSQL and PostgreSQL-to-DSQL schema conversion, foreign key constraints, OCC retry patterns, ORM migration (Django/EF Core/Hibernate/Rails/SQLAlchemy), DDL operations, query plan explainability, system diagnostics via CloudWatch AAS, SQL compatibility validation, and bulk data loading. Triggers on phrases like: DSQL, Aurora DSQL, distributed SQL database, serverless PostgreSQL-compatible database, migrate to DSQL, DSQL query plan, DSQL EXPLAIN ANALYZE, DSQL ENUM, DSQL foreign key, DSQL OCC retry, DSQL multi-region, DSQL JSONB, DSQL GIN index, load into DSQL, load CSV into DSQL, bulk load DSQL, aurora-dsql-loader, DSQL slow, DSQL performance, DSQL wait events, DSQL AAS."
description: "Build with Aurora DSQL — manage schemas, execute queries, handle migrations, diagnose query plans, diagnose cluster performance, load data, and develop applications with a serverless, distributed SQL database. Covers IAM auth, multi-tenant patterns, MySQL-to-DSQL and PostgreSQL-to-DSQL schema conversion, foreign key constraints, OCC retry patterns, ORM migration (Django/Drizzle/EF Core/Hibernate/Rails/SQLAlchemy), DDL operations, query plan explainability, system diagnostics via CloudWatch AAS, SQL compatibility validation, and bulk data loading. Triggers on phrases like: DSQL, Aurora DSQL, distributed SQL database, serverless PostgreSQL-compatible database, migrate to DSQL, DSQL query plan, DSQL EXPLAIN ANALYZE, DSQL ENUM, DSQL foreign key, DSQL OCC retry, DSQL Drizzle, Drizzle ORM with DSQL, DSQL multi-region, DSQL JSONB, DSQL GIN index, load into DSQL, load CSV into DSQL, bulk load DSQL, aurora-dsql-loader, DSQL slow, DSQL performance, DSQL wait events, DSQL AAS."
license: Apache-2.0
metadata:
tags: aws, aurora, dsql, distributed-sql, distributed, distributed-database, database, serverless, serverless-database, postgresql, postgres, sql, schema, migration, multi-tenant, iam-auth, aurora-dsql, mcp, orm, enum, foreign-key, occ-retry, django, ef-core, dotnet, csharp, hibernate, rails, multi-region, schema-conversion, type-mapping, data-loading, system-diagnostics, wait-events, aas, performance, cloudwatch
tags: aws, aurora, dsql, distributed-sql, distributed, distributed-database, database, serverless, serverless-database, postgresql, postgres, sql, schema, migration, multi-tenant, iam-auth, aurora-dsql, mcp, orm, enum, foreign-key, occ-retry, django, drizzle, typescript, ef-core, dotnet, csharp, hibernate, rails, multi-region, schema-conversion, type-mapping, data-loading, system-diagnostics, wait-events, aas, performance, cloudwatch
---

# Amazon Aurora DSQL Skill
Expand Down Expand Up @@ -65,9 +65,9 @@ Load these files as needed for detailed guidance:

### ORM Guides:

| Reference | When to Load | Contains |
| ----------------------------------------------------------- | ------------------------- | ------------------------------------------------------------------------ |
| [orm-guides/overview.md](references/orm-guides/overview.md) | Migrating any ORM to DSQL | Adapter names, key gotchas for Django/EF Core/Hibernate/Rails/SQLAlchemy |
| Reference | When to Load | Contains |
| ----------------------------------------------------------- | ------------------------- | -------------------------------------------------------------------------------- |
| [orm-guides/overview.md](references/orm-guides/overview.md) | Migrating any ORM to DSQL | Adapter names, key gotchas for Django/Drizzle/EF Core/Hibernate/Rails/SQLAlchemy |

### Data Loading:

Expand Down Expand Up @@ -271,7 +271,7 @@ MUST load [query-plan/workflow.md](references/query-plan/workflow.md) at entry

MUST load [pg-migrations/type-mapping.md](references/pg-migrations/type-mapping.md), [pg-migrations/schema-objects.md](references/pg-migrations/schema-objects.md), and [foreign-keys.md](references/foreign-keys.md). Run `dsql_lint(fix=true)` first for mechanical fixes, preserve foreign-key relationships, translate unsupported source syntax or options, then apply semantic conversions from the pg-migrations references for unfixable diagnostics and patterns the linter cannot handle. Re-lint the final output before deploying.

### Workflow 11: ORM Migration (Django/EF Core/Hibernate/Rails/SQLAlchemy)
### Workflow 11: ORM Migration (Django/Drizzle/EF Core/Hibernate/Rails/SQLAlchemy)

Load [orm-guides/overview.md](references/orm-guides/overview.md) for adapter names and framework-specific gotchas.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Part of [DSQL Development Guide](../development-guide.md).

## Database Connectivity Tools

DSQL has many tools for connecting including 12 database drivers, 4 ORM libraries, and 4 specialized adapters
DSQL has many tools for connecting including 12 database drivers, 4 ORM libraries, and 5 specialized adapters
Comment thread
praba2210 marked this conversation as resolved.
across various languages as listed in the [programming guide](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/aws-sdks.html). PREFER using connectors, drivers, ORM libraries, and adapters.

### Database Drivers
Expand Down Expand Up @@ -49,6 +49,7 @@ Specific extensions that make existing ORMs work with Aurora DSQL:
| **Java** | Hibernate | [Aurora DSQL Hibernate Adapter](https://github.com/awslabs/aurora-dsql-hibernate/) |
| **Python** | Django | [Aurora DSQL Django Adapter](https://github.com/awslabs/aurora-dsql-django/) |
| **Python** | SQLAlchemy | [Aurora DSQL SQLAlchemy Adapter](https://github.com/awslabs/aurora-dsql-sqlalchemy/) |
| **TypeScript** | Drizzle | [Aurora DSQL Drizzle Adapter](https://github.com/awslabs/aurora-dsql-orms/tree/main/node/drizzle) |
Comment thread
praba2210 marked this conversation as resolved.

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ Concrete example (from `dsql_lint(sql="CREATE INDEX idx ON t (c);", fix=true)`):

## Workflow: Validate & Migrate SQL to DSQL

Use for any SQL that was not composed by the agent itself from skill knowledge — including user-pasted SQL, migration files, ORM output (Django, Rails, Prisma, TypeORM, Sequelize, SQLAlchemy), pg_dump exports, and hand-written schemas. Applies to DDL and schema-mutating DML; do **not** lint ad-hoc read-only `SELECT`s.
Use for any SQL that was not composed by the agent itself from skill knowledge — including user-pasted SQL, migration files, ORM output (Django, Rails, Prisma, Drizzle, TypeORM, Sequelize, SQLAlchemy), pg_dump exports, and hand-written schemas. Applies to DDL and schema-mutating DML; do **not** lint ad-hoc read-only `SELECT`s.

1. Obtain source SQL from user (migration file, ORM output, schema dump, or inline SQL). `dsql_lint` accepts multi-statement SQL in a single call — pass the whole batch.
2. Run `dsql_lint(sql=source_sql, fix=true)`. Default to `fix=true` for any migration scenario; use `fix=false` only when the user explicitly asked for validation-only output, or when re-verifying manually rewritten SQL.
Expand Down Expand Up @@ -94,6 +94,13 @@ Use for any SQL that was not composed by the agent itself from skill knowledge
- **Django:** Run `python manage.py sqlmigrate <app> <migration>` to get raw SQL, then lint.
- **Rails (6.1+):** Set `config.active_record.schema_format = :sql`, then run `rails db:schema:dump` (legacy `db:structure:dump` still works in older Rails). Lint the generated `db/structure.sql`.
- **Prisma:** Use `prisma migrate diff --from-empty --to-schema-datamodel ./prisma/schema.prisma --script` to emit SQL to stdout, then lint.
- **Drizzle:** Run `npx aurora-dsql-drizzle generate --out ./drizzle -- --config drizzle.config.ts` — the adapter runs `drizzle-kit generate` and pipes each statement through `dsql-lint`, preserving `--> statement-breakpoint` markers (`transform` and `lint` run those steps alone). It lints one statement at a time, so cross-statement fixes do not apply — see [orm-guides/overview.md](orm-guides/overview.md).
`aurora-dsql-drizzle` exit codes differ by mode — `generate` and `transform` run `dsql-lint --fix`, `lint` only reports:
- `generate`, `transform`: `0` clean — apply the migration; `1` some diagnostics need a hand-written rewrite — see Handling Unfixable Errors below; `3` fixed with advisories — review the flagged statements, then apply. One advisory statement sets `3` for the whole run, e.g. `NOT VALID` added to a foreign key.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should preserve the workflow’s existing acknowledgement requirement for fixed_with_warning output. “Review, then apply” could allow an agent to proceed without user acknowledgement.

Suggested replacement:

3 fixed with advisories — show the diagnostics and transformed SQL to the user, obtain acknowledgement, then apply.

- `lint`: `0` clean; `1` any diagnostic was reported, including advisories `transform` fixes on its own — run `transform` before hand-writing a rewrite. `lint` never returns `3` and never writes SQL.
- `1` also covers failures that say nothing about the SQL — bad arguments, a missing input file, a `drizzle-kit` error, or a missing `dsql-lint` binary. Read the `Error:` line on stderr and confirm diagnostics were printed before treating `1` as a verdict on the SQL.

When an advisory adds `NOT VALID` to a foreign key, follow it with `ALTER TABLE ASYNC ... VALIDATE CONSTRAINT` in its own chunk — precede the statement with a `--> statement-breakpoint` marker — then verify the job.
- **TypeORM/Sequelize:** Generate migration SQL to a file, then lint.
- **SQLAlchemy:** Compile DDL without executing — e.g., `for table in metadata.tables.values(): print(CreateTable(table).compile(engine))`. Do **not** call `metadata.create_all(engine)` with a real engine — it executes the DDL before lint. Alternatively use `create_mock_engine` to capture DDL.

Expand Down
6 changes: 6 additions & 0 deletions plugins/databases-on-aws/skills/dsql/references/language.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,12 @@ or [postgres-js](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/SECTIO
- [DSQL postgres-js preferred example](https://github.com/aws-samples/aurora-dsql-samples/blob/main/javascript/postgres-js/src/example_preferred.js)
- See [aurora-dsql-samples/javascript/postgres-js](https://github.com/aws-samples/aurora-dsql-samples/tree/main/javascript/postgres-js)

#### Drizzle

- Adapter: [@aws/aurora-dsql-drizzle](https://github.com/awslabs/aurora-dsql-orms/tree/main/node/drizzle) (rides `drizzle-orm/node-postgres`; no custom dialect)
- `drizzle({ connection: { host, user, region }, schema })` — `user` is required, so a connection never defaults to `admin`
- Use `db.transactionWithRetry()` for OCC retry and the package's `migrate()` (one DDL per transaction) instead of the stock `drizzle-orm/node-postgres` migrator — see [orm-guides/overview.md](orm-guides/overview.md) for framework gotchas

#### Prisma

- Custom `directUrl` with token refresh middleware
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,20 +11,23 @@ Across adapters, inline foreign keys in `CREATE TABLE` use DSQL foreign-key synt
Post-creation foreign keys **MUST** use `ADD CONSTRAINT ... NOT VALID`, followed by
`ALTER TABLE ASYNC ... VALIDATE CONSTRAINT` and terminal job verification.

For existing tables, emit that sequence through the framework's raw-SQL migration hook:
`RunSQL` (Django), `migrationBuilder.Sql` (EF Core), Flyway/Liquibase (Hibernate), `execute`
(Rails), or `op.execute` (Alembic/SQLAlchemy). For tenant-scoped composite FKs, use raw DDL in
Django and Rails; EF Core, Hibernate, and SQLAlchemy provide composite relationship mappings.
For existing tables, emit that sequence through the framework's raw-SQL migration mechanism:
`RunSQL` (Django), the generated `.sql` migration file (Drizzle), `migrationBuilder.Sql` (EF Core),
Flyway/Liquibase (Hibernate), `execute` (Rails), or `op.execute` (Alembic/SQLAlchemy). For
tenant-scoped composite FKs, use raw DDL in Django and Rails; Drizzle
(`foreignKey({ columns, foreignColumns })`), EF Core, Hibernate, and SQLAlchemy provide composite
relationship mappings.

## Adapters

| Framework | Adapter | Install |
| ---------- | --------------------------------------- | ------------------------------------------------------------ |
| Django | `aurora_dsql_django` | `pip install aurora-dsql-django boto3` |
| EF Core | `Amazon.AuroraDsql.EntityFrameworkCore` | `dotnet add package Amazon.AuroraDsql.EntityFrameworkCore` |
| Hibernate | `aurora-dsql-hibernate-dialect` | `software.amazon.dsql:aurora-dsql-hibernate-dialect` (Maven) |
| Rails | Standard `pg` gem + `aws-sdk-dsql` | `gem 'pg'` + `gem 'aws-sdk-dsql'` |
| SQLAlchemy | `aurora_dsql_sqlalchemy` | `pip install aurora-dsql-sqlalchemy boto3` |
| Framework | Adapter | Install |
| ---------- | --------------------------------------- | ------------------------------------------------------------------------------------ |
| Django | `aurora_dsql_django` | `pip install aurora-dsql-django boto3` |
| Drizzle | `@aws/aurora-dsql-drizzle` | `npm install @aws/aurora-dsql-drizzle drizzle-orm pg` + `npm install -D drizzle-kit` |
| EF Core | `Amazon.AuroraDsql.EntityFrameworkCore` | `dotnet add package Amazon.AuroraDsql.EntityFrameworkCore` |
| Hibernate | `aurora-dsql-hibernate-dialect` | `software.amazon.dsql:aurora-dsql-hibernate-dialect` (Maven) |
| Rails | Standard `pg` gem + `aws-sdk-dsql` | `gem 'pg'` + `gem 'aws-sdk-dsql'` |
| SQLAlchemy | `aurora_dsql_sqlalchemy` | `pip install aurora-dsql-sqlalchemy boto3` |

## Key Gotchas Per Framework

Expand All @@ -39,6 +42,26 @@ Django and Rails; EF Core, Hibernate, and SQLAlchemy provide composite relations
| AutoField | Replace with `UUIDField(primary_key=True, default=uuid.uuid4)` |
| ForeignKey | Keep `ForeignKey`; the DSQL backend creates database constraints for new tables |

### Drizzle (TypeScript)

Requires `drizzle-orm` `^0.45` (0.45.x — the peer range pins the minor), `pg` 8+, Node.js 20+, and

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The foreign-key behavior below requires @aws/dsql-lint 0.2.17+, but the published adapter accepts versions starting at 0.2.14. Please add the version floor here so existing lockfiles cannot silently remove foreign keys.

Suggested language:

Requires drizzle-orm ^0.45, pg 8+, Node.js 20+, drizzle-kit as a dev dependency, and @aws/dsql-lint 0.2.17+. Update older lockfiles before generating migrations.

`drizzle-kit` as a dev dependency. The adapter rides `drizzle-orm/node-postgres` over the DSQL
Comment thread
praba2210 marked this conversation as resolved.
node-postgres connector — there is no custom dialect.

| Issue | Fix |
| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Setup | `drizzle({ connection: { host, user, region }, schema })` from `@aws/aurora-dsql-drizzle` — IAM auth and TLS come from the connector, and `region` is optional (inferred from the host). Pass an existing pool with `drizzle({ client: pool, schema })`; `db.$client` exposes it |
| `user` | REQUIRED — there is no default, so a connection never lands on `admin` by omission. Pass a database role scoped to what the app needs |
| OCC retry | `db.transactionWithRetry(cb)` re-runs the transaction on SQLSTATE `40001`. The callback MUST be idempotent — it re-runs on every attempt. Nested `tx.transaction()` fails fast. Defaults: `maxRetries` 3, `baseDelayMs` 50, `maxDelayMs` 5000 |
| Retry exhaustion | Throws `AwsDsqlRetryExhaustedError` with the last conflict on `.cause`. Pass overrides in the third argument, after the transaction config: `transactionWithRetry(cb, undefined, { maxRetries: 5, onRetry: (err, attempt, max) => ... })` |
| Migrations | Use `migrate()` from `@aws/aurora-dsql-drizzle`, NOT the stock `drizzle-orm/node-postgres` migrator — the stock one sends every statement in a single transaction. `getMigrationStatus(db, config)` reports applied vs. pending |
| Generate | `npx aurora-dsql-drizzle generate --out ./drizzle -- --config drizzle.config.ts` runs `drizzle-kit generate`, then rewrites each statement with `dsql-lint`. Review and commit the result — see [dsql-lint.md](../dsql-lint.md) |
| `SERIAL` | The transform rewrites it as `BIGINT ... GENERATED BY DEFAULT AS IDENTITY (CACHE 1)`. Review the DDL for the wider type, and let the identity column supply values — replace any `nextval()`/`setval()`/`currval()` calls that named the old `SERIAL` sequence |
| Breakpoints | Keep Drizzle Kit's `breakpoints: true` (the default). The adapter applies one statement per `--> statement-breakpoint` marker, and `migrate()` stops with `success: false` on any chunk holding more than one statement. Give every hand-added statement its own marker — `transform` neither inserts one nor flags the chunk, so the failure surfaces at apply time |
| Identity columns | Define them in the table definition. The transform lints statements one at a time, so `ALTER COLUMN ... ADD GENERATED ... AS IDENTITY` is reported unfixable — merge it into the `CREATE TABLE` by hand if it is already generated |
| Foreign keys | The transform adds `NOT VALID` to foreign keys created with `ALTER TABLE`. Add `ALTER TABLE ASYNC ... VALIDATE CONSTRAINT` after it, preceded by its own `--> statement-breakpoint` marker; `migrate()` waits for that job |
| Resume | A statement and its tracking row are separate commits. If a run dies between them the statement is applied but untracked, and re-running fails "already exists" (`drizzle-kit` omits `IF NOT EXISTS`) — `migrate()` names the statement |

### EF Core (.NET)

Requires .NET 8.0+, EF Core 9.0.7+, and `Amazon.AuroraDsql.Npgsql` 1.1.0+.
Expand Down
Loading
Loading