Firehose for Stellar

Quick start with Firehose for Stellar can be found in the official Firehose docs. Here some quick links to it:

• Firehose Overview
• Concepts & Architectures
  • Components
  • Data Flow
  • Data Storage
  • Design Principles

Running the Firehose fetcher

Two fetcher backends are available. Both emit the same pbbstream.Block shape; check proto/sf/stellar/type/v1/block.proto for the payload schema.

Captive-core is the supported backend going forward. The RPC poller is kept for compatibility but is no longer actively developed — new deployments should use captive-core.

Captive-core backend (recommended)

Spawns a stellar-core subprocess and streams ledgers from it.

firestellar fetch captive-core {FIRST_STREAMABLE_BLOCK} \
  --stellar-core-bin /usr/bin/stellar-core \
  --stellar-core-network mainnet \
  --state-dir {STATE_DIR}

RPC backend (legacy)

Streams ledgers from a Stellar RPC endpoint. Maintenance-only — prefer captive-core for new work.

firestellar fetch rpc {FIRST_STREAMABLE_BLOCK} --endpoints {STELLAR_RPC_ENDPOINT} --state-dir {STATE_DIR}

Resume behavior (--state-dir / --ignore-cursor)

Both backends persist the last fired block to {STATE_DIR}/cursor.json after each successful emission. On restart, the fetcher resumes at last_fired_block + 1 instead of replaying from {FIRST_STREAMABLE_BLOCK}.

• --state-dir — directory holding cursor.json. Default: /data/work (both backends). Pass an empty string to disable persistence.
• --ignore-cursor — ignore any persisted cursor.json and start fresh from {FIRST_STREAMABLE_BLOCK}. Use this when running under a supervisor (e.g. firecore reader-node) that already tracks downstream state and passes the correct start block on restart.

The cursor schema is shared between the two backends, so a single state directory can be reused if you switch backends.

Releasing

Releases run in CI, do not run sfreleaser release locally.

1. Open a PR retitling the ## Unreleased section of CHANGELOG.md to ## vX.Y.Z. A comment on the PR shows the release notes and the commits going out.
2. Merge it. .github/workflows/draft-release.yml creates a draft GitHub release with the binaries.
3. Review the draft and publish it. Publishing creates the tag, which pushes the Docker images, and .github/workflows/publish-release.yml updates the Homebrew formula.

workflow_dispatch on release.yml runs the full cross-compile build without publishing anything.

Contributing

For more information, please read the CONTRIBUTING.md file.