Skip to content

feat(datastore): idempotent (TOUCH-like) bulk relationship import#3195

Open
poucet wants to merge 1 commit into
authzed:mainfrom
poucet:datastore/bulk-import-touch-semantics
Open

feat(datastore): idempotent (TOUCH-like) bulk relationship import#3195
poucet wants to merge 1 commit into
authzed:mainfrom
poucet:datastore/bulk-import-touch-semantics

Conversation

@poucet

@poucet poucet commented Jun 26, 2026

Copy link
Copy Markdown

Summary

ImportBulkRelationships routes through the datastore BulkLoad, which used strict CREATE semantics: importing a relationship that already existed aborted the whole load with an AlreadyExists error.

This PR makes BulkLoad idempotent (TOUCH-like) across all datastores — already-existing relationships are silently skipped instead of failing the load, so re-imports are idempotent.

Changes per datastore

  • Postgres / CockroachDB: replace the shared pgxcommon.BulkLoad CopyFrom implementation with batched INSERT ... ON CONFLICT DO NOTHING. Bare ON CONFLICT DO NOTHING works for both (Postgres infers the uq_relation_tuple_living_xid partial unique index — the same arbiter WriteRelationships TOUCH relies on; CockroachDB infers pk_relation_tuple). Rows are batched under the 65535 bind-parameter wire limit.
  • MySQL: use INSERT IGNORE (already an idiom in this codebase).
  • memdb: skip relationships that already exist while scanning the source.
  • Spanner: use an InsertOrUpdate mutation instead of Insert.

Returned count (num_loaded)

BulkLoad now returns the number of relationships actually inserted (excluding skipped duplicates) for Postgres, CockroachDB, MySQL, and memdb. Spanner cannot cheaply distinguish new from existing rows (mutations are applied blindly at commit), so it continues to report the number processed; this is documented at the call site.

Tests

  • Add BulkUploadIdempotentTest to the shared datastore conformance suite (AllWithExceptions), so it runs for every datastore. It asserts the no-duplicate invariant via queries (universally true) plus the initial-load count, and covers re-load, superset-load, and same-transaction duplicate cases.
  • Remove the now-obsolete BulkUploadAlreadyExistsError / BulkUploadAlreadyExistsSameCallError tests and the bulk-load portion of CreateAlreadyExistingTest, which asserted the removed error-on-duplicate behavior.

Behavior / compatibility notes

  • Behavior change: a bulk import containing relationships that already exist no longer returns AlreadyExists; those rows are skipped on every datastore.
  • Semantics: "idempotent no-op" TOUCH (skip if exists). It does not update caveat/expiration on already-existing rows for the skip-based backends (PG/CRDB/MySQL/memdb); Spanner's InsertOrUpdate does overwrite, a minor cross-store difference noted in code.
  • Performance (PG/CRDB): trades raw COPY throughput for upsert-style batched inserts. This is inherent to conflict tolerance — COPY cannot express ON CONFLICT, and a COPY-into-temp-table approach was rejected because CockroachDB temp tables are experimental/off-by-default.

Testing performed

Per-datastore bulk suites + TestCreateAlreadyExisting pass on memdb, Postgres (both configs), CockroachDB (integrity + non-integrity), MySQL, and Spanner. Full memdb suite and the service-level ImportBulkRelationships test also pass.

@poucet poucet requested a review from a team as a code owner June 26, 2026 12:11
@github-actions github-actions Bot added area/datastore Affects the storage system area/tooling Affects the dev or user toolchain (e.g. tests, ci, build tools) labels Jun 26, 2026
@github-actions

Copy link
Copy Markdown

CLA Assistant Lite bot:
Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request

@poucet poucet force-pushed the datastore/bulk-import-touch-semantics branch from b1aeb0a to cb278e1 Compare June 26, 2026 12:13
ImportBulkRelationships routes through the datastore BulkLoad, which used
strict CREATE semantics: importing a relationship that already existed
aborted the load with an AlreadyExists error. Make BulkLoad idempotent
(TOUCH-like) across all datastores, so already-existing relationships are
silently skipped rather than failing the load, which makes re-imports
idempotent.

Per datastore:
- postgres/crdb: replace the shared CopyFrom-based BulkLoad with batched
  INSERT ... ON CONFLICT DO NOTHING. Bare ON CONFLICT DO NOTHING works for
  both backends (Postgres infers the uq_relation_tuple_living_xid partial
  unique index; CockroachDB infers pk_relation_tuple). Rows are batched to
  stay under the 65535 bind-parameter wire limit.
- mysql: use INSERT IGNORE.
- memdb: skip relationships that already exist when scanning the source.
- spanner: use an InsertOrUpdate mutation instead of Insert.

Returned count semantics: BulkLoad now returns the number of relationships
actually inserted (excluding skipped duplicates) for postgres, crdb, mysql,
and memdb. Spanner cannot cheaply distinguish new from existing rows
(mutations are applied blindly at commit), so it continues to report the
number processed; this is documented at the call site.

Testing:
- Add BulkUploadIdempotentTest to the shared datastore suite, asserting the
  no-duplicate invariant via queries (holds for every datastore) plus the
  initial-load count.
- Remove the now-obsolete BulkUploadAlreadyExistsError /
  BulkUploadAlreadyExistsSameCallError tests and the bulk-load portion of
  CreateAlreadyExistingTest, which asserted the removed error-on-duplicate
  behavior.
@poucet poucet force-pushed the datastore/bulk-import-touch-semantics branch from cb278e1 to ed2bf8c Compare June 26, 2026 12:39
@github-actions github-actions Bot removed the area/tooling Affects the dev or user toolchain (e.g. tests, ci, build tools) label Jun 26, 2026
@poucet poucet changed the title feat(datastore): idempotent (TOUCH-like) bulk relationship import for Postgres and CockroachDB feat(datastore): idempotent (TOUCH-like) bulk relationship import Jun 26, 2026
@tstirrat15

Copy link
Copy Markdown
Contributor

We considered this when writing the *Bulk APIs in the first place, and decided against it because it wasn't meaningfully different enough from using WriteRels with a Touch semantic in a loop. We use COPY FROM and friends because they're meaningfully faster if you're able to work around the conflict semantics.

For zed, we wrote the RetryableClient which wraps around this behavior to retry batches that do have conflicts according to a configurable semantic, and this is the way we'd recommend approaching a bulk load with potential conflicts.

If you aren't using golang, we'd welcome a PR to the client library you're using. I've put up a draft PR of one in Java here: authzed/authzed-java#174, though with the caveat that I'm not well-versed in writing idiomatic Java and I'd welcome feedback on it if that's the direction y'all would be going.

@poucet

poucet commented Jun 29, 2026

Copy link
Copy Markdown
Author

What about making this an option on the ImportBulkRelationships request?

If I want to load a lot of data into my spicedb, then it would be nice if that load was idempotent if it failed half-way through. TOUCH semantics are perfect for that.

@tstirrat15

Copy link
Copy Markdown
Contributor

How would that option work? Are you talking about the individual messages that are sent over the wire or the overall request or?

I agree that TOUCH semantics would be nice here; we'd have already used them in this API if the datastores that SpiceDB uses exposed that functionality to us.

@poucet

poucet commented Jun 29, 2026

Copy link
Copy Markdown
Author

I was considering a singular boolean option at the request level to determine whether to go with the existing mechanism or the one I introduced in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/datastore Affects the storage system

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants