feat(docker): add Docker Compose stack for runtime services by eliteprox · Pull Request #35 · livepeer/clearinghouse

eliteprox · 2026-06-17T23:16:41Z

Summary

Adds the fundamental docker compose stack including go-livepeer remote signer, Kafka and an OpenMeter/Benthos collector with parsing config for kafka events

Excludes identity webhook

Docker Compose — Kafka/Redpanda, go-livepeer remote signer, OpenMeter/Benthos collector

Quick start

cp deploy/.env.example deploy/.env

docker compose -f deploy/docker-compose.yml --env-file deploy/.env up -d --build \
  kafka remote-signer openmeter-collector

Test plan

Add REMOTE_SIGNER_WEBHOOK_URL
Signer port 8081 published; CLI port 4935 not published

Closes #23
Closes #24

Kafka, remote signer, and OpenMeter collector with local compose targets.

Changed the volume binding for the signer service to use a configurable data directory. Updated the entrypoint script to correct the remote signer webhook header format for compatibility.

Drop AUTH0_PUBLIC_CLIENT_ID from deploy env/README now that the stacked identity-webhook uses API-key auth. Remove PR delivery notes from README.

…ntation Refactor the entrypoint script to use a more concise argument passing method and correct the format of the `WEBHOOK_SECRET` in the README to include the required syntax for the Authorization header.

eliteprox · 2026-06-22T16:51:59Z

@rickstaa This PR is ready for review. Let me know if you see any potential optimizations, I think this set of changes are pretty clear now

linear-code · 2026-06-23T22:29:01Z

ENG-135

ENG-134

ENG-121

rickstaa

Hey @eliteprox, I reviewed this pull request, and overall it looks strong. I was also able to confirm that the signer + Kafka part of the stack is running and posting messages to the OpenMeter endpoint.

I left a few minor comments. Once you’ve addressed those, and any others you think are important, this can be merged.

…ollector

…dpanda integration Replaced references to Apache Kafka with Redpanda in both the README and docker-compose.yml. Clarified the architecture and identity validation process for the remote signer container.

…cript Included comments in the entrypoint.sh to clarify the usage of Redpanda in development mode, highlighting the lack of security and persistence features, and linking to the relevant issue for tracking.

Updated comments in docker-compose.yml to specify that the keystore and .eth-password should be copied into deploy/data before the first run. Revised README to streamline the description of the bootstrap process for meters and features.

Updated docker-compose.yml and collector.yaml to ensure ETH_USD_PRICE is a required environment variable, removing the default value. Revised README to reflect this change, indicating that the ETH/USD rate must be provided explicitly.

…etup Added KAFKA_ADVERTISED_ADDR and KAFKA_BROKERS to the .env.example file for better Kafka integration. Updated docker-compose.yml to utilize these environment variables, simplifying the configuration for the Kafka and signer services. Revised README to document the new environment variables and their purposes.

Introduced SIGNER_ETH_KEYSTORE_PATH in docker-compose.yml to allow specifying an optional keystore directory or keyfile path. Updated entrypoint.sh to pass this new environment variable to the signer service. Revised README to document the new variable and its purpose.

Introduced SIGNER_DATA_DIR in .env.example to specify the host directory for signer state and keystore. Updated README to document this new variable and added details for SIGNER_REMOTE_DISCOVERY and ORCH_WEBHOOK_URL, enhancing clarity on their usage.

Revised .env.example to clarify OpenMeter URLs for Konnect and self-hosted setups. Updated docker-compose.yml to remove the deprecated OPENMETER_URL. Updated README to provide clearer instructions for setting up the environment.

…cker Compose setup for clearinghouse stack Added a new docker-compose.yml file to define the services for the clearinghouse stack, including Redpanda, go-livepeer remote signer, and OpenMeter collector. Introduced a .gitignore file to exclude sensitive environment files and created example .env files for each service to facilitate configuration. Updated README to outline the components and setup instructions for the stack.

Enhanced the README to clarify the identity and authorization process for the remote signer, including instructions for local alive checks without a webhook. Updated .env.example to indicate that leaving REMOTE_SIGNER_WEBHOOK_URL empty allows starting the signer without webhook authorization, while emphasizing the necessity of this variable in production.

eliteprox mentioned this pull request Jun 17, 2026

feat: self-contained identity webhook + collector price oracle and metering fixes #36

Merged

9 tasks

eliteprox changed the title ~~feat(deploy): Docker Compose stack on Go bootstrap~~ feat(deploy): Add Signer, Kafka and Collector docker compose with manual deploy scripts Jun 17, 2026

eliteprox force-pushed the feat/deploy-stack-go-bootstrap branch from 15162ff to 6cd8f07 Compare June 18, 2026 22:09

eliteprox changed the title ~~feat(deploy): Add Signer, Kafka and Collector docker compose with manual deploy scripts~~ feat(deploy): add Docker Compose stack for runtime services Jun 18, 2026

eliteprox changed the base branch from feat/go-bootstrap-cli to main June 18, 2026 22:10

eliteprox changed the base branch from main to feat/go-bootstrap-cli June 18, 2026 22:14

eliteprox force-pushed the feat/deploy-stack-go-bootstrap branch 2 times, most recently from 5d5ca7d to f45f9be Compare June 18, 2026 22:15

feat(deploy): add Docker Compose stack for runtime services

f176e2e

Kafka, remote signer, and OpenMeter collector with local compose targets.

eliteprox force-pushed the feat/deploy-stack-go-bootstrap branch from f45f9be to f176e2e Compare June 18, 2026 22:16

eliteprox changed the base branch from feat/go-bootstrap-cli to main June 18, 2026 22:16

fix(deploy): update Docker Compose configuration and entrypoint script

116502e

Changed the volume binding for the signer service to use a configurable data directory. Updated the entrypoint script to correct the remote signer webhook header format for compatibility.

eliteprox requested a review from rickstaa June 19, 2026 01:01

eliteprox added 2 commits June 22, 2026 11:50

chore(deploy): remove unused Auth0 env var and tidy identity docs

53b1865

Drop AUTH0_PUBLIC_CLIENT_ID from deploy env/README now that the stacked identity-webhook uses API-key auth. Remove PR delivery notes from README.

fix(deploy): update remote signer entrypoint script and README docume…

e11ec1e

…ntation Refactor the entrypoint script to use a more concise argument passing method and correct the format of the `WEBHOOK_SECRET` in the README to include the required syntax for the Authorization header.

eliteprox mentioned this pull request Jun 23, 2026

feat(deploy): Docker stack + OpenMeter bootstrap + Railway CI (MVP 0) #22

Closed

7 tasks

eliteprox added this to the Deliverable 1 - NodeJS Auth Webhook milestone Jun 23, 2026

This was referenced Jun 23, 2026

Scaffold clearinghouse example repo consuming builder-sdk v0.4.1 #21

Closed

Scaffold clearinghouse example repo consuming builder-sdk (on the #35/#36 foundation) #1

Closed

deploy/: go-livepeer remote-signer container #23

Closed

rickstaa approved these changes Jun 24, 2026

View reviewed changes

eliteprox added 4 commits June 24, 2026 14:44

docker(collector): pin docker image version for openmeterio/benthos-c…

6132276

…ollector

docker(signer): pin go-livepeer image to v0.8.11

1005d32

chore(deploy): add .gitignore to exclude environment file

71eeb0b

docs(deploy): update README and docker-compose comments to reflect Re…

e93230b

…dpanda integration Replaced references to Apache Kafka with Redpanda in both the README and docker-compose.yml. Clarified the architecture and identity validation process for the remote signer container.

eliteprox mentioned this pull request Jun 24, 2026

Remove --mode dev-container flag on HA prod deployments #43

Open

eliteprox added 2 commits June 24, 2026 15:31

docs(deploy): add notes for Redpanda development mode in entrypoint s…

80648df

…cript Included comments in the entrypoint.sh to clarify the usage of Redpanda in development mode, highlighting the lack of security and persistence features, and linking to the relevant issue for tracking.

eliteprox added 2 commits June 24, 2026 16:19

eliteprox force-pushed the feat/deploy-stack-go-bootstrap branch from f237cc2 to 99d56b0 Compare June 24, 2026 20:33

eliteprox force-pushed the feat/deploy-stack-go-bootstrap branch from 61fd9cd to 6d1858a Compare June 24, 2026 20:39

eliteprox added 4 commits June 24, 2026 16:46

feat(deploy): update OpenMeter configuration

622de5a

Revised .env.example to clarify OpenMeter URLs for Konnect and self-hosted setups. Updated docker-compose.yml to remove the deprecated OPENMETER_URL. Updated README to provide clearer instructions for setting up the environment.

eliteprox changed the title ~~feat(deploy): add Docker Compose stack for runtime services~~ feat(docker): add Docker Compose stack for runtime services Jun 24, 2026

eliteprox merged commit 30545dc into main Jun 24, 2026

rickstaa deleted the feat/deploy-stack-go-bootstrap branch June 25, 2026 08:01

eliteprox linked an issue Jun 30, 2026 that may be closed by this pull request

Benthos collector — create_signed_ticket → CloudEvents → OpenMeter #7

Closed

eliteprox mentioned this pull request Jun 30, 2026

Benthos collector — create_signed_ticket → CloudEvents → OpenMeter #7

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(docker): add Docker Compose stack for runtime services#35

feat(docker): add Docker Compose stack for runtime services#35
eliteprox merged 17 commits into
mainfrom
feat/deploy-stack-go-bootstrap

eliteprox commented Jun 17, 2026 •

edited

Loading

Uh oh!

eliteprox commented Jun 22, 2026

Uh oh!

linear-code Bot commented Jun 23, 2026

Uh oh!

rickstaa left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

eliteprox commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Quick start

Test plan

Uh oh!

eliteprox commented Jun 22, 2026

Uh oh!

linear-code Bot commented Jun 23, 2026

Uh oh!

rickstaa left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

eliteprox commented Jun 17, 2026 •

edited

Loading

rickstaa left a comment •

edited

Loading