Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 34 additions & 25 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -1201,23 +1201,23 @@ against the local SQLite store, and is where future issuer actions (e.g. `mint`,
receipts to a corroborated destination through the Turnkey signer, driving the
receipt-moving engine (see "Receipt custody"). The destination is stated by
exactly one of two mutually exclusive flags: `--to-configured-orchestrator`
reads `[orchestrator].address` from `--config` (the cutover path — the
orchestrator address is never typed), while `--to <ADDRESS>` states an
explicit destination (the wallet-rotation path). `--to` naming the configured
orchestrator address is refused: the cutover path must state it through the
config flag, so a hand-typed orchestrator address never enters the flow.
Either way the stated address only becomes reachable as a transfer destination
through the corroboration witness described under "Receipt custody". Before
anything is signed the command verifies the deployment hold is armed (the hold
file present and the readiness marker absent — the engine's projection
rebuilds must not race a running service), verifies the bot wallet's native
balance covers a fixed transfer-gas ceiling at the current gas price (no
per-transaction estimate — estimating a transfer of receipts the destination
does not hold yet would revert), corroborates the destination, and then
prompts with the asset, vault, holder, destination and its corroborated kind,
and the tracked receipt count — the operator confirms what was proven, not
what was typed. A re-run after a completed move reports the already-migrated
observation distinctly and submits nothing.
reads the `--network`'s `[orchestrator.addresses]` entry from `--config` (the
cutover path — the orchestrator address is never typed), while
`--to <ADDRESS>` states an explicit destination (the wallet-rotation path).
`--to` naming the configured orchestrator address is refused: the cutover path
must state it through the config flag, so a hand-typed orchestrator address
never enters the flow. Either way the stated address only becomes reachable as
a transfer destination through the corroboration witness described under
"Receipt custody". Before anything is signed the command verifies the
deployment hold is armed (the hold file present and the readiness marker
absent — the engine's projection rebuilds must not race a running service),
verifies the bot wallet's native balance covers a fixed transfer-gas ceiling
at the current gas price (no per-transaction estimate — estimating a transfer
of receipts the destination does not hold yet would revert), corroborates the
destination, and then prompts with the asset, vault, holder, destination and
its corroborated kind, and the tracked receipt count — the operator confirms
what was proven, not what was typed. A re-run after a completed move reports
the already-migrated observation distinctly and submits nothing.
- `issuer confirm-custody <UNDERLYING>` — verifies on-chain that the Turnkey bot
wallet holds exactly every tracked receipt balance for the asset's vault, then
records it as the inventory's custody holder. The rollback counterpart of
Expand Down Expand Up @@ -1404,10 +1404,11 @@ The orchestrator onboarding and custody subcommands (`orchestrator-preflight`,
`confirm-custody`) take the same network flags and require the `TURNKEY_*`
group: the facts they verify or establish are keyed to the Turnkey bot wallet,
so a local-key signer is refused. All but `confirm-custody` also take `--config`
(the TOML configuration file; its `[orchestrator].address` is the only source of
the orchestrator address — never typed); `confirm-custody` involves no
orchestrator address at all. See `docs/runbooks/orchestrator-onboarding.md` for
the ordered onboarding and per-asset cutover procedures.
(the TOML configuration file; its per-network `[orchestrator.addresses]` map is
the only source of the orchestrator address — never typed); `confirm-custody`
involves no orchestrator address at all. See
`docs/runbooks/orchestrator-onboarding.md` for the ordered onboarding and
per-asset cutover procedures.

### Receipt custody

Expand Down Expand Up @@ -2037,6 +2038,10 @@ File"): an asset whose `[assets.<UNDERLYING>]` table sets
`vault_mode = "orchestrator"` routes its mints and burns through the
orchestrator; an asset without an override takes
`[orchestrator].default_vault_mode`, which itself defaults to `"vault_direct"`.
The mode is keyed by symbol alone (it applies on every network the asset is
listed on); the orchestrator **address** is keyed by network via
`[orchestrator.addresses]` — each chain carries its own deployment — and is
resolved at the operation's anchoring point from the operation's own network.
The mapping is loaded once at startup (changing it is a config change + restart,
like any other deploy-time setting) and threaded to the two call sites that
resolve it — `MintServices` and `BurnManager`, each of which resolves a given
Expand Down Expand Up @@ -4454,16 +4459,20 @@ default.

```toml
[orchestrator]
# ST0xOrchestrator contract address. Required when any asset resolves to
# orchestrator mode; rejected as a startup error if that is the case and it
# is missing or malformed.
address = "0x..."
# Mode for assets without a per-asset override below:
# "vault_direct" (the default when omitted) | "orchestrator".
# The full-rollout end state sets this to "orchestrator" and drops the
# per-asset overrides, so newly onboarded assets default to the orchestrator.
default_vault_mode = "vault_direct"

# ST0xOrchestrator contract addresses, one per network — each chain carries
# its own deployment. Keys are network wire names (base | ethereum |
# hyperevm). Required when any asset resolves to orchestrator mode: startup
Comment thread
rouzwelt marked this conversation as resolved.
# then demands an entry for EVERY configured chain, and rejects unknown
# network keys and missing, malformed, or zero addresses.
[orchestrator.addresses]
base = "0x..."

# Per-asset override, keyed by underlying symbol. During the pilot exactly one
# asset carries this; every other asset stays on the default.
[assets.RKLB]
Expand Down
21 changes: 13 additions & 8 deletions config.example.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,21 +6,26 @@
#
# Parsing is strict — unknown keys and invalid vault_mode strings are startup
# errors, and any asset resolving to orchestrator mode while
# [orchestrator].address is missing or malformed is a startup error.
# [orchestrator.addresses] is missing or malformed is a startup error.

[orchestrator]
# ST0xOrchestrator contract address. Required when any asset resolves to
# orchestrator mode; rejected as a startup error if that is the case and it
# is missing, malformed, or the zero address.
address = "0x1234567890abcdef1234567890abcdef12345678"

# Mode for assets without a per-asset override below:
# "vault_direct" (the default when omitted) | "orchestrator".
# The full-rollout end state sets this to "orchestrator" and drops the
# per-asset overrides, so newly onboarded assets default to the orchestrator.
default_vault_mode = "vault_direct"

# Per-asset override, keyed by underlying symbol. During the pilot exactly one
# asset carries this; every other asset stays on the default.
# ST0xOrchestrator contract addresses, one per network — each chain carries
# its own deployment. Keys are network wire names: base | ethereum | hyperevm.
# Required when any asset resolves to orchestrator mode: at startup every
# configured chain must have an entry (a missing, malformed, zero-address, or
# unknown-network entry is a startup error).
[orchestrator.addresses]
base = "0x1234567890abcdef1234567890abcdef12345678"

# Per-asset override, keyed by underlying symbol. The mode is keyed by symbol
# alone — it applies on every network the asset is listed on; the address is
# what varies per network. During the pilot exactly one asset carries this;
# every other asset stays on the default.
[assets.RKLB]
vault_mode = "orchestrator"
2 changes: 1 addition & 1 deletion config.prod.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@
#
# Parsing is strict — unknown keys and invalid vault_mode strings are startup
# errors, and any asset resolving to orchestrator mode while
# [orchestrator].address is missing or malformed is a startup error.
# [orchestrator.addresses] is missing or malformed is a startup error.
2 changes: 1 addition & 1 deletion config.staging.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@
#
# Parsing is strict — unknown keys and invalid vault_mode strings are startup
# errors, and any asset resolving to orchestrator mode while
# [orchestrator].address is missing or malformed is a startup error.
# [orchestrator.addresses] is missing or malformed is a startup error.
47 changes: 26 additions & 21 deletions docs/runbooks/orchestrator-onboarding.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,20 +43,25 @@ on-chain resolution. The one address argument in this document,
configured orchestrator address, and is guarded by the kind-aware corroboration
witness (see SPEC "Receipt custody").

## 1. Ship the orchestrator address in the config (stays dark)
## 1. Ship the orchestrator addresses in the config (stays dark)

Add to `config.prod.toml`:

```toml
[orchestrator]
address = "0x…" # from st0x.deploy
[orchestrator.addresses]
base = "0x…" # from st0x.deploy; one entry per network — each chain has
# its own orchestrator deployment
```

Do **not** add any `[assets.<SYM>]` section — with no `vault_mode` overrides
every asset stays vault-direct, so this deploys dark. Parsing is strict (unknown
keys and a malformed or zero address are startup errors, even while dark).
Verify locally with `cargo run --bin validate-config`, then deploy. The config
file is baked into the systemd unit (`CONFIG=<nix store path>`, see
keys, unknown network names, and malformed or zero addresses are startup errors,
even while dark), and once any asset resolves to orchestrator mode, startup
requires an entry for **every** configured chain. Every `issuer` verification
below runs per `--network` against that network's entry, and an asset's cutover
(steps 7–14) runs per chain it is listed on. Verify locally with
`cargo run --bin validate-config`, then deploy. The config file is baked into
the systemd unit (`CONFIG=<nix store path>`, see
`nix/upgradeable-services.nix`). Every command below passes `--config "$CONFIG"`
— the unit's own value — so the CLI provably validates and approves against the
exact file the running service resolves, never a stray local copy.
Expand Down Expand Up @@ -122,13 +127,13 @@ One-time unlimited ERC-20 approval, bot wallet → orchestrator, on the asset's
vault share token, signed by Turnkey after an explicit confirmation. Before
sending, the command verifies the configured address answers as an orchestrator
(interface reads plus a healthy `vaultLogicIsExpected()`), so a typo'd or stale
`[orchestrator].address` is refused rather than granted an unlimited allowance.
Idempotent: a re-run reports "already unlimited" and sends nothing, so batching
every asset's approval early is safe — approvals are inert until the asset's
`vault_mode` flips. Success is re-verified by an on-chain allowance read. When
this step actually submits, the transaction is also live proof that the policy's
`approve` allowance works; the idempotent no-op path proves nothing new — step
4's signing proof covers `approve` in that case.
`[orchestrator.addresses]` entry is refused rather than granted an unlimited
allowance. Idempotent: a re-run reports "already unlimited" and sends nothing,
so batching every asset's approval early is safe — approvals are inert until the
asset's `vault_mode` flips. Success is re-verified by an on-chain allowance
read. When this step actually submits, the transaction is also live proof that
the policy's `approve` allowance works; the idempotent no-op path proves nothing
new — step 4's signing proof covers `approve` in that case.

Record each executed approval in the table below.

Expand Down Expand Up @@ -239,14 +244,14 @@ issuer move-receipts <SYM> \
--network base --chain-id 8453 --rpc-url "$RPC_URL"
```

The destination is read from `[orchestrator].address` — never typed — and
corroborated as an ERC-1155-receiving contract before anything is signed. The
command prompts with the asset, vault, holder, destination and its corroborated
kind, and the tracked receipt count. A vault tracking more than 14 receipts
moves in multiple bounded transactions, each verified before the next. A re-run
after any interruption is safe: an interrupted move resumes with only the
remaining receipts, and a completed move reports "already migrated" and submits
nothing.
The destination is read from the `--network`'s `[orchestrator.addresses]` entry
— never typed — and corroborated as an ERC-1155-receiving contract before
anything is signed. The command prompts with the asset, vault, holder,
destination and its corroborated kind, and the tracked receipt count. A vault
tracking more than 14 receipts moves in multiple bounded transactions, each
verified before the next. A re-run after any interruption is safe: an
interrupted move resumes with only the remaining receipts, and a completed move
reports "already migrated" and submits nothing.

## 11. Verify the move

Expand Down
92 changes: 69 additions & 23 deletions src/admin.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2231,7 +2231,17 @@ pub(crate) async fn orchestrator_health(
let mut orchestrators_seen: Vec<(Network, Address)> = Vec::new();

for asset in &enabled_assets {
match config.vault_mode_for(&asset.underlying) {
let mode = config
.vault_mode_for(&asset.underlying, asset.network)
.map_err(|err| {
error!(target: "admin", network = %asset.network,
underlying = %asset.underlying,
error = %err,
"Orchestrator address missing for asset's network"
);
Status::InternalServerError
})?;
match mode {
Comment thread
rouzwelt marked this conversation as resolved.
VaultMode::VaultDirect => {
assets.push(AssetVaultModeStatus {
underlying: asset.underlying.clone(),
Expand Down Expand Up @@ -3020,7 +3030,7 @@ mod tests {
AlpacaError, AlpacaService, MintCallbackRequest, RedeemRequest,
RedeemRequestStatus, RedeemResponse, TokenizationRequest,
};
use crate::config::{VaultMode, VaultModeConfig};
use crate::config::{VaultMode, VaultModeConfig, VaultModeKind};
use crate::mint::test_utils::{
TestHarness, network_vault_services, test_config,
};
Expand Down Expand Up @@ -6865,11 +6875,9 @@ mod tests {
seed_enabled_asset(&pool, "TSLA", tsla_vault).await;

let vault_mode_config = VaultModeConfig::new(
HashMap::from([(
"AAPL".to_string(),
VaultMode::Orchestrator { address: orchestrator },
)]),
VaultMode::VaultDirect,
HashMap::from([("AAPL".to_string(), VaultModeKind::Orchestrator)]),
VaultModeKind::VaultDirect,
HashMap::from([(Network::Base, orchestrator)]),
);
let vault_service: Arc<dyn VaultService> = Arc::new(
MockVaultService::new_success()
Expand Down Expand Up @@ -6928,11 +6936,9 @@ mod tests {
.await;

let vault_mode_config = VaultModeConfig::new(
HashMap::from([(
"AAPL".to_string(),
VaultMode::Orchestrator { address: orchestrator },
)]),
VaultMode::VaultDirect,
HashMap::from([("AAPL".to_string(), VaultModeKind::Orchestrator)]),
VaultModeKind::VaultDirect,
HashMap::from([(Network::Base, orchestrator)]),
);
let vault_service: Arc<dyn VaultService> = Arc::new(
MockVaultService::new_success().with_vault_logic_expected(false),
Expand All @@ -6953,6 +6959,49 @@ mod tests {
);
}

/// An orchestrator-kind asset whose network has no
/// `[orchestrator.addresses]` entry must surface as a 500 with an ERROR
/// log — never silently report as vault-direct. Unreachable in a
/// validated deploy (the startup cross-check requires an entry per
/// configured chain), so this pins the fail-loud behavior of the gap.
#[traced_test]
#[tokio::test]
async fn orchestrator_health_missing_address_is_an_internal_error() {
let pool = setup_pool().await;
seed_enabled_asset(
&pool,
"AAPL",
address!("0xaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"),
)
.await;

let vault_mode_config = VaultModeConfig::new(
HashMap::from([("AAPL".to_string(), VaultModeKind::Orchestrator)]),
VaultModeKind::VaultDirect,
HashMap::new(),
);
let vault_service: Arc<dyn VaultService> =
Arc::new(MockVaultService::new_success());
let rocket = orchestrator_health_rocket(
pool,
health_config(vault_mode_config),
vault_service,
);

let (status, _body) = dispatch_orchestrator_health(rocket, true).await;

assert_eq!(
status,
Status::InternalServerError,
"a missing per-network address must fail loudly, not report \
vault-direct"
);
assert!(logs_contain_at!(
Level::ERROR,
&["Orchestrator address missing for asset's network"]
));
}

#[traced_test]
#[tokio::test]
async fn orchestrator_health_dedupes_shared_orchestrator() {
Expand All @@ -6974,7 +7023,8 @@ mod tests {

let vault_mode_config = VaultModeConfig::new(
HashMap::new(),
VaultMode::Orchestrator { address: orchestrator },
VaultModeKind::Orchestrator,
HashMap::from([(Network::Base, orchestrator)]),
);
let mock = Arc::new(MockVaultService::new_success());
let vault_service: Arc<dyn VaultService> = mock.clone();
Expand Down Expand Up @@ -7015,11 +7065,9 @@ mod tests {
.await;

let vault_mode_config = VaultModeConfig::new(
HashMap::from([(
"AAPL".to_string(),
VaultMode::Orchestrator { address: orchestrator },
)]),
VaultMode::VaultDirect,
HashMap::from([("AAPL".to_string(), VaultModeKind::Orchestrator)]),
VaultModeKind::VaultDirect,
HashMap::from([(Network::Base, orchestrator)]),
);
let vault_service: Arc<dyn VaultService> =
Arc::new(MockVaultService::new_success().with_vault_logic_error());
Expand Down Expand Up @@ -7066,11 +7114,9 @@ mod tests {
.await;

let vault_mode_config = VaultModeConfig::new(
HashMap::from([(
"AAPL".to_string(),
VaultMode::Orchestrator { address: orchestrator },
)]),
VaultMode::VaultDirect,
HashMap::from([("AAPL".to_string(), VaultModeKind::Orchestrator)]),
VaultModeKind::VaultDirect,
HashMap::from([(Network::Base, orchestrator)]),
);
let vault_service: Arc<dyn VaultService> = Arc::new(
MockVaultService::new_success().with_next_burn_receipt_id_error(),
Expand Down
Loading
Loading