Skip to content

feat(OpenMeter): add kongctl bootstrap scripts#51

Closed
eliteprox wants to merge 5 commits into
feat/identity-webhookfrom
feat/openmeter-provisioning
Closed

feat(OpenMeter): add kongctl bootstrap scripts#51
eliteprox wants to merge 5 commits into
feat/identity-webhookfrom
feat/openmeter-provisioning

Conversation

@eliteprox

@eliteprox eliteprox commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

Makes the OpenMeter collector path end-to-end usable against Konnect and replaces the Go SDK provisioner from #39 with thin, idempotent kongctl bootstrap scripts.

Collector fixes (openmeter-collector/collector.yaml):

  1. Emit billable_usd_micros — the billable meter (SUM on $.billable_usd_micros) rejected every create_signed_ticket event with "invalid event: missing value property". The collector now emits billable_usd_micros as an interim passthrough equal to network_fee_usd_micros so the meter validates and accumulates. Phase-2 markup (network × pipeline/model multiplier) is deferred.
  2. Emit compound CloudEvent subject — OpenMeter attributes usage by exact subject match; customers are keyed (and subject-keyed) by client_id:usage_subject. A bare usage_subject never matched, producing "no customer found for event subject". The collector now sets subject = auth_id (compound) while data.client_id / data.usage_subject remain for per-tenant/user meter dimensions.

Provisioning (openmeter-collector/provision/):

  • catalog.json — meters, features, and plan key (data, not code)
  • bootstrap.sh / bootstrap.ps1 — cross-platform scripts using kongctl api (no native meter resource yet; see Kong/kongctl#1334)
  • Subcommands: catalog | customer <client_id> <external_user_id> | all
  • Customers keyed + subject-keyed by compound client_id:external_user_id; scripts never mutate subject_keys (OpenMeter blocks that on subscribed customers)

Closes #

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Refactor / tech debt
  • Documentation
  • CI / tooling

Affected component(s)

  • builder-sdk (core)
  • Signer proxy
  • Usage API / helper methods
  • Kafka collection endpoint
  • Storage / SQL schema
  • Docker container
  • OIDC / auth (pymthouse)

Integration mode

  • Hosted
  • On-prem / self-hosted
  • Both / N/A

How was this tested?

Verified against live Konnect:

  • bootstrap.sh catalog — idempotent; meters/features created or reported up to date
  • bootstrap.sh customer demo-client demo-user — demo customer provisioned with compound subject key
  • Ingested a create_signed_ticket CloudEvent with compound subject demo-client:demo-user — both billable_usd_micros and network_fee_usd_micros meters return values (usage attributes to the customer)

Checklist

  • My branch is rebased on the latest default branch
  • I added/updated tests covering my change
  • I updated documentation (README / docs) where relevant
  • I updated CHANGELOG_PENDING.md if this is a user-facing change
  • No new required dependencies were introduced (or they are justified below)
  • Secrets, signer keys, and client IDs are not committed

Notes for reviewers

  • Stacks on feat/identity-webhook; review collector identity contract alongside webhook auth_id shape.
  • Root README.md still references the Go clearinghouse-bootstrap CLI in the bootstrap section — follow-up to point at openmeter-collector/provision/ (or remove the Go CLI reference once feat(collector): implement customer upsert functionality in OpenMeter collector #39 is fully superseded).
  • Root README.md identity contract still lists subject = usage_subject; the collector and provision README now document compound subject = client_id:usage_subject. Align in a follow-up or as part of this PR if preferred.
  • billable_usd_micros == network_fee_usd_micros is intentional until phase-2 markup lands.

@eliteprox eliteprox changed the title Feat/openmeter provisioning OpenMeter: fix collector attribution/billing and add kongctl bootstrap scripts Jun 29, 2026
eliteprox and others added 2 commits June 29, 2026 17:50
The billable_usd_micros meter (SUM on $.billable_usd_micros) rejected every
create_signed_ticket event with "invalid event: missing value property" because
the collector never emitted that field. Emit it as an interim passthrough equal
to network_fee_usd_micros so the meter validates and accumulates; phase-2 markup
(network × pipeline/model multiplier) is applied later. Update the README's
two-meter section accordingly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…omer

OpenMeter attributes usage by exact subject match and forbids changing a
customer's subject_keys once it has an active subscription. Customers are keyed
(and subject-keyed) by the compound client_id:usage_subject, so a bare
usage_subject subject never matches — producing "no customer found for event
subject". Emit subject = auth_id (compound) instead of the bare usage_subject;
data.client_id / data.usage_subject remain for per-tenant/user meter dimensions.

Verified against live Konnect: a compound-subject create_signed_ticket event
attributes to demo-client:demo-user (billable + network meters return values).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@eliteprox eliteprox changed the title OpenMeter: fix collector attribution/billing and add kongctl bootstrap scripts feat(OpenMeter): add kongctl bootstrap scripts Jun 29, 2026
eliteprox and others added 2 commits June 29, 2026 17:50
Idempotent provisioning of meters, features, plan check, and tenant customers
via `kongctl api` (kongctl has no native meter resource yet, Kong/kongctl#1334),
replacing the Go SDK provisioner from #39 with thin cross-platform scripts and a
JSON catalog.

- catalog.json   — meters/features/plan_key (data, not code)
- bootstrap.sh   — Linux/Darwin; bootstrap.ps1 — Windows
- subcommands: catalog | customer <client_id> <external_user_id> | all
- customers keyed + subject-keyed by compound client_id:external_user_id;
  never mutates subject_keys (OpenMeter blocks that on subscribed customers)
- verified against live Konnect: catalog idempotent, demo customer up to date

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…gement

- Added support for loading environment variables from a specified .env file in both PowerShell and Bash scripts.
- Improved error handling for missing Konnect PAT and added checks for meter and feature associations.
- Introduced new functions for managing features and plans, including creation, validation, and publishing of plans.
- Updated catalog.json to include a structured plan definition with phases and rate cards, replacing the previous plan_key approach.

This update streamlines the provisioning process and ensures that features are correctly linked to their respective meters, enhancing the overall functionality of the OpenMeter provisioning scripts.
@eliteprox
eliteprox force-pushed the feat/openmeter-provisioning branch from 43024a1 to 8b643c9 Compare June 29, 2026 21:52
@eliteprox

Copy link
Copy Markdown
Collaborator Author

Superceeded by #53

@eliteprox eliteprox closed this Jun 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant