Skip to content

Migrate content buckets to hierarchical namespace #539

Description

@caugner

What information was incorrect, unhelpful, or incomplete?

The content buckets use flat namespace. HNS offers:

  • Up to 8 times higher initial read and write QPS limits.
  • Atomic folder renames, which can publish or replace a release prefix without copying each object.

These performance benefits matter for serving and deploying large content builds. HNS is a prerequisite for the release-specific prefixes proposed in #528. Existing buckets cannot be upgraded to HNS.

What did you expect to see?

  • Provision new HNS buckets for the content buckets.
  • Copy the existing state.
  • Update Dex's configuration and deployment workflows to use the new buckets.

The review bucket already uses HNS and does not need migration.

Do you have any supporting links, references, or citations?

Do you have anything more you want to share?

HNS requires uniform bucket-level access and does not support Object Versioning, Object Retention Lock, object holds, Bucket Lock, object-level ACLs, cross-bucket replication, or bucket relocations with write downtime. Check whether these constraints affect the current buckets during migration planning.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs triageTriage needed by staff and/or partners. Automatically applied when an issue is opened.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions