Use Yano's automatic sparse backfill and KES defaults that outlive a devnet by satran004 · Pull Request #193 · bloxbean/yaci-devkit

satran004 · 2026-09-17T05:49:43Z

Targets main and includes #192 (Ogmios 7.0.0 / Kupo 2.12.0, commit 10cffc0), so both land together in one squash. Close #192 once this is merged. Main (#188, #189, #190) is merged in, and this PR pins Yano 0.1.0-pre17 and Yaci Store 2.0.4 (see Updates below).

Summary

Two devnet defaults learned from Yano's own devnet configuration. Neither is visible on a short-epoch devnet; both get worse as epoch length grows, which is exactly the configuration long-epoch work is trying to make viable.

Automatic sparse backfill

Yano's catch-up from the shifted genesis to wall clock places one empty block per slot by default, so create time grows with epoch length and the Haskell relay has to copy the whole chain.

Yano pre15 can derive the spacing itself (bloxbean/yano#127, via #126 → main on 09-11, released in pre15 on 09-12). It places only the blocks a validating node needs:

gaps stay inside the forecast window (3k/f) so the relay can validate every header
epoch-boundary blocks are kept, for nonces
a final block lands at the catch-up target so the producer resumes at wall clock

Everything in between was empty filler. Upstream measured 584 ms vs 3436 ms catch-up, and ~15 blocks for three 1200-slot epochs.

DevKit passes yano.block-producer.backfill-block-interval-slots=0 and lets Yano decide, rather than deriving an interval itself. Yano knows the forecast window, the epoch nonce schedule and the producer's eligibility headroom; a number computed out here is a heuristic that silently drifts when those rules change. Yano also validates its own value and fails loudly:

Backfill block interval must be non-negative (0 = automatic), got:
Backfill interval must be below forecast window of
Sparse backfill found no eligible block within the forecast window;
  reduce the interval or increase the devnet producer stake

Override with yano.backfill.block.interval.slots: auto (default), dense for one block per slot, or an integer. An unusable value falls back to automatic, not dense — falling back to the slow default would hide the misconfiguration.

KES defaults

	Before	After	Mainnet
`slotsPerKESPeriod`	129,600	10,000,000	129,600
`maxKESEvolutions`	60	62	62

The node derives the current KES period as slot / slotsPerKESPeriod, and an opcert expires once elapsed periods exceed maxKESEvolutions. The companion bootstrap shifts genesis back by whole epochs and then catches up to wall clock, so the chain starts at a high slot number:

14-day epochs (1,209,600 slots), 3-epoch shift = 3,628,800 slots
3,628,800 / 129,600 = 28 of 60 evolutions consumed before the first real block

And the share grows with epoch length. The large period keeps a devnet inside KES period 0 for its whole life. Note maxKESEvolutions 60 → 62 moves toward mainnet; 60 was the outlier.

Trade-off: a DevKit devnet no longer exercises KES rotation. Documented alongside the values to set for mainnet-like behaviour. Scoped to the devnet genesis defaults only — the privnet spec/ templates used by PrivNetService, which don't time-travel, keep the mainnet values.

Relationship to #191

This supersedes #191, which derives (3k/f)/4 in DevKit and passes an explicit integer. That PR was written against yano#124, before Yano gained automatic mode — its code only sets the property when > 1, so it can never reach 0. Its dependency note ("no effect until DevKit pins a Yano release") is also now stale: pre15 carries the property.

Suggest closing #191 in favour of this; the property naming and docs from it are reused here.

A note for whoever integrates rjharmon's branch

YanoConfigBuilder.build() is now at six parameters, and #189 and #191 each add one more. Merged naively that becomes seven positional arguments with adjacent booleans — worth collapsing into a small options record at integration time rather than letting each PR add one.

Updates (2026-09-26)

Three more commits on top of the original change:

1. Bump Yano to 0.1.0-pre17 and Yaci Store to 2.0.3 (00eda1d)

Yano pre17 for the CLI download and the Docker image. The Docker image (Earthfile YANO_TAG/YANO_VERSION) was still pinned to 0.1.0-pre6, which predates the backfill property, so without this the image would have kept dense backfill.
Nothing DevKit depends on changed between pre16 and pre17 (property keys, BackfillPolicy, devnet profile).
Yaci Store 2.0.3: native binaries from rel-native-2.0.3, jar from v2.0.3. The Docker image now builds Store from the v2.0.3 tag instead of release/2.0.1_devkit. The four devkit-only PRs on that branch (Enable both submit-api and ogimios service for yaci-store all profile, so that controllers are available in YS all native image yaci-store#965, Normalize direct-start genesis protocol parameters yaci-store#967, Align devnet genesis AdaPot bookkeeping with Haskell ledger yaci-store#977, Add missing jooq classes to graalvm config yaci-store#978) are all contained in v2.0.3; yaci-store#977 is there with later refinements.

2. Keep capturing process output when a line cannot be processed (0880c8f)

Existing bug: ProcessUtil did String.format("[%s] " + line, ...). After any stop/start the node prints Progress: 95.65% while reopening its ChainDB, the format call threw, and the reader thread died. From then on the logs command stayed frozen at startup, and the tee'd node.log stopped growing once the pipe filled.
The line is now appended as data, and ProcessStream skips a line whose consumer throws instead of ending the read loop (unit test added).

3. Let cardano-node write and rotate its own node.log (d60417a)

The node scripts piped stdout through tee -a node.log, so the rotation block in configuration.json never applied and node.log grew without bound (measured ~85 KB/min, ~120 MB/day at 1s blocks).
configuration.json adds a FileSK scribe (5 MB x 10 files, 24h); tee is removed from all five node scripts.
node.log is now a link to the newest timestamped file, so use tail -F. The file has no ANSI colour codes. The Node configuration: … dump the node prints before its logger starts is only on stdout (the CLI logs command) now. Nothing in the repo reads node.log; the logs command reads the process stdout.

Measurements (Yano pre16, same build, only `yano.backfill.block.interval.slots` changed)

Default devnet (600-slot epochs, companion), seconds from create-node to the first Haskell block:

Mode	Backfill blocks	Relay sync	Ready
dense	1,800	~2.2s	16.1s (16.2 / 16.0)
auto	9	~0.4s	12.7s (12.1 / 13.2)

The fixed cost (Yano start, governance txs, node restart) dominates at 600-slot epochs; dense cost grows with epoch length, auto stays a handful of blocks per epoch.

Docker verification (native parity)

earthly +cli-docker (native CLI, Store native built from v2.0.3 = commit 381dce2, Yano pre17 linux-arm64), run with config/env + config/node.properties like docker-compose:

companion bootstrap on pre17, relay synced at height 16, Yano stopped, Haskell forging
Yaci Store 2.0.3 synced to tip (lag 0) and keeping up; protocol version 11 with PV11 cost models 332/332/350
Ogmios 7.0.0 connected, 100% synced, Conway
topup transaction indexed by Store (/utxos, /balance, /amounts)
node.log written by the node itself; logs command current after startup

Not covered: amd64 image, viewer image, multi-node (ports taken on the test machine), e2e SDK suites.

Merged with `main` (2026-09-26)

Resolved in 5182ce4; two follow-up commits on top:

YanoConfigBuilder.build() now takes the history dir (Update Ogmios to 7.0.0 / Kupo to 2.12.0 and fix devnet startup issues #192), Slot leader time travel below f1 #189's slot-leader flag and this PR's backfill interval (seven parameters; still worth collapsing into an options record later, see note below). YanoService keeps both yano.backfill.block.interval.slots and yano.past.time.travel.slot.leader.mode.
application.properties, both node.properties and the two docs pages keep every section; node.properties keeps slotsPerKESPeriod=10000000 next to genesis: parse stabilityWindowFactor as a double and document it #190's stabilityWindowFactor.
87676ad Default traceChainSyncClient to false: Slot leader time travel below f1 #189 turned the node's chain-sync client tracing on by default; on multi-node devnets it logs every header from every peer for the devnet's lifetime. The setting stays, off by default, and the docs point to it for a relay that won't sync.
5aef858 Drops the stale slotsPerKESPeriod 129600 / maxKESEvolutions 60 entries at the end of the node-configuration docs, which contradicted the new defaults.

Verified on the merged branch with Yano pre17 (companion, Store off):

Run	Backfill	Relay	After handover
default (f = 1)	`Sparse backfill: one block every 299 slots`, 9 blocks	synced at height 16	+20 blocks in 20s; `TraceChainSyncClient: false`; `node.log` → timestamped file
`--block-time 2` (f = 0.5)	#189 auto-enabled slot-leader mode: `Sparse slot-leader backfill: first eligible slot every 150 slots`	synced at height 17, 0 `VRFLeaderValueTooBig`	+6 blocks in 23 slots (≈ f = 0.5)

The second row is the first run of #189 and this PR's automatic spacing together.

Yaci Store 2.0.4 (2026-09-30)

8dafa6a bumps Yaci Store from 2.0.3 to 2.0.4, which closes the Blockfrost API gaps the SDKs hit:

download.properties: native binaries from rel-native-2.0.4, jar from v2.0.4. Both releases use the same asset names as 2.0.3.
Both Earthfiles: STORE_NATIVE_BRANCH="v2.0.4".

Verified on a companion devnet (Yano pre17, --epoch-length 40) running the 2.0.4 macOS arm64 n2c native binary. The SDK e2e suites were run through each SDK's Blockfrost provider, with every SDK updated to its latest version, and all 13 tests passed:

Suite	Tests
Evolution SDK	protocol_params, payment, plutus_v3
Lucid Evolution	payment, plutus_v2, plutus_v3
MeshJS	payment, payment_splitter_plutusV3
PyCardano	payment, plutus_v3
cardano-client-lib	Payment, MintToken, PlutusV3

Not re-run for 2.0.4: the Docker image and the yano-only mode, which uses the yaci-store-all binary.

Test plan

Suggested squash title: Yano 0.1.0-pre17 sparse backfill, KES defaults, Ogmios 7.0.0 / Kupo 2.12.0, Yaci Store 2.0.4 and node.log rotation (#193)

🤖 Generated with Claude Code

Move Ogmios and Kupo back to the CardanoSolutions upstream and drop the PV11 stopgap that bundled Linux x86_64 IntersectMBO builds into the arm64 image. Both components now ship native binaries for linux/amd64, linux/arm64 and macOS arm64, so the Docker images no longer need x86_64 emulation and a native macOS install works. Ogmios / Kupo - Bump to Ogmios 7.0.0 and Kupo 2.12.0 from CardanoSolutions (.tar.gz -> .zip) - Use real aarch64 assets for arm64 instead of emulated x86_64 ones - Consolidate the duplicated OS/arch resolution into one helper that fails fast with an actionable message on unpublished platforms (macOS x86_64, Windows) instead of 404-ing - Flatten the nested archive layout the macOS zips use (v7.0.0/bin/ogmios) so the binary always lands at <home>/bin/<name> - Handle the Kupo tag/asset version mismatch (tag v2.12, asset v2.12.0) Ogmios as the Yaci Store tx evaluator - Enable Ogmios by default (ogmios_enabled=true) and use it to evaluate Plutus script costs, falling back to the embedded Scalus evaluator - Wait for Ogmios to answer on /health before committing to the ogmios evaluator mode. The mode is baked into the Store process at startup and never re-evaluated, so a short-lived Ogmios would otherwise leave the Store unable to evaluate any script for the lifetime of the devnet - Skip Ogmios/Kupo in yano-only mode; they need the Haskell node's n2c socket - Don't let an Ogmios failure abort the startup chain before Yaci Store runs - Treat enableOgmios as optional in the admin API so omitting it keeps the configured default rather than forcing Ogmios off Fix prometheus.port never being applied The property was read, stored on ClusterInfo and put into the Mustache values map, but templates/configuration.json hardcoded 12798 with no placeholder, so the value was silently dropped. A second cardano-node on the host holding 12798 made the devnet node die at bind with the error going to stderr, which node.sh does not capture. Disable the Yano DuckLake projection archive Yano pre15's devnet profile enables the projection archive by default. DevKit reads nothing from Yano's history API, and the archive defaults to ./history relative to the working directory - the shared yano home - so it outlived create-node -o and then failed the startup identity and coverage guards. Turn it off and pin the dir inside the node folder so it is cleaned up with the cluster if it is ever re-enabled. Also bumps Yano to 0.1.0-pre15. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…devnet Two devnet defaults learned from Yano's own devnet configuration, both of which only bite as epoch length grows. Automatic sparse backfill Yano's catch-up from the shifted genesis to wall clock places one empty block per slot by default, so devnet create time grows with epoch length and the Haskell relay has to copy the whole chain. Yano 0.1.0-pre15 can derive the spacing itself from genesis (bloxbean/yano#127): it keeps every gap inside the forecast window a validating node can span, retains the epoch-boundary blocks needed for nonces, and places a final block at the catch-up target. Everything in between was empty filler. DevKit now passes yano.block-producer.backfill-block-interval-slots=0 and lets Yano decide, rather than deriving an interval itself. Yano knows the forecast window, the epoch nonce schedule and the producer's eligibility headroom; a number computed out here would be a heuristic that silently drifts when those rules change, and Yano validates its own value and fails loudly instead. yano.backfill.block.interval.slots overrides it: auto (default), dense for one block per slot, or an integer to force the interval. An unusable value falls back to automatic rather than to dense, which is the slow one. KES defaults slotsPerKESPeriod moves from the mainnet 129600 to 10000000, and maxKESEvolutions from 60 to the mainnet 62. The node derives the current KES period as slot / slotsPerKESPeriod, and the companion bootstrap shifts genesis back by whole epochs before catching up to wall clock, so the chain starts at a high slot. At 129600 a long-epoch devnet burns a large share of its evolutions before producing its first real block - a three epoch shift on 14 day epochs is 28 of the 60 - and the share grows with epoch length. The large period keeps a devnet inside KES period 0 for its whole life. The trade-off is that a DevKit devnet no longer exercises KES rotation, which is documented alongside the values to set to get mainnet-like behaviour back. Only the devnet genesis defaults change; the privnet spec templates, which do not time travel, keep the mainnet values. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

- Yano 0.1.0-pre17 for both the CLI download (download.properties) and the Docker image (Earthfile YANO_TAG/YANO_VERSION, download-yano.sh default). The Docker image was still pinned to 0.1.0-pre6, which predates the backfill interval property. - Yaci Store 2.0.3: CLI native binaries from rel-native-2.0.3 and the jar from v2.0.3. The Docker image now builds the Store native variants from the v2.0.3 tag instead of release/2.0.1_devkit; the devkit-only fixes on that branch (#965, #967, #977, #978) are all contained in v2.0.3. Verified with a native Docker build (Store commit 381dce2 = v2.0.3, Yano pre17 linux-arm64): companion bootstrap, Store synced to tip, PV11 cost models enacted, Ogmios 7.0.0 synced, and a topup transaction indexed by Store. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

ProcessUtil built each captured line with String.format("[%s] " + line, ...), so the line itself was used as a format string. After a restart cardano-node prints "Progress: 95.65%" while it reopens its ChainDB; the format call threw, the reader thread died, and nothing drained the node's stdout any more. The `logs` command then stayed frozen at startup and the tee'd node.log stopped growing once the pipe filled. - append the line as data instead of formatting it - ProcessStream skips a line whose consumer throws instead of ending the read loop Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

The node scripts piped stdout through `tee -a node.log`, so the rotation settings in configuration.json never applied (they only cover file scribes) and node.log grew without bound, about 120 MB/day at 1s blocks. - configuration.json adds a FileSK scribe for node.log next to stdout; it uses the existing rotation block (5 MB per file, 10 files, 24h) - node.sh, node-2, node-3, node-bp.sh and node-relay.sh no longer pipe through tee The node now writes timestamped files with node.log as a link to the newest, so use `tail -F`. The CLI `logs` command reads the process stdout and is unaffected. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Brings in #188 (relay sync wait by progress), #189 (slot-leader time travel below f=1) and #190 (stabilityWindowFactor as a double). - YanoConfigBuilder.build() takes the history dir, #189's slot-leader flag and the backfill interval; YanoService keeps both settings and passes both - application.properties, both node.properties and the node-modes / node-configuration docs keep every section; the node.properties examples keep slotsPerKESPeriod=10000000 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

#189 turned the node's chain-sync client tracing on by default. On a single-node companion devnet the node has no upstream peer after the handover, so it costs little, but on multi-node devnets it logs every header each node receives from each peer for the life of the devnet. Keep the setting, default it to off, and point to it in the docs for diagnosing a relay that will not sync. The unrendered configuration.yaml and spec/config.json templates go back to false as well; configuration.json takes the value from traceChainSyncClient. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

The page still listed slotsPerKESPeriod 129600 and maxKESEvolutions 60 near the end, contradicting the "maxKESEvolutions and slotsPerKESPeriod" section above it, which documents the current defaults (62 / 10000000) and the trade-off. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Native binaries from rel-native-2.0.4, jar from v2.0.4, and the Docker image builds Store from the v2.0.4 tag. 2.0.4 closes the Blockfrost API gaps the SDKs hit. Verified on a companion devnet with the 2.0.4 macOS arm64 n2c binary: all SDK e2e suites pass (Evolution SDK, Lucid Evolution, MeshJS, PyCardano, cardano-client-lib; 13 tests) through each SDK's Blockfrost provider, with the SDKs at their latest versions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Every suite builds, signs and submits through its SDK's Blockfrost provider against Yaci Store, so they check that Store works as a drop-in Blockfrost backend. - New suites: Evolution SDK, PyCardano and cardano-client-lib (JBang) - Lucid Evolution 0.6.5 (explicit slotConfig for the Custom network, via devnet.ts) and MeshJS 1.9.1 (BlockfrostProvider; Plutus tests spend only the UTxO they locked) - run-sdk-tests.sh runs all suites and prints a PASS/FAIL table. --update moves each SDK to its latest version, --restart recreates the devnet, --store-jar / --store-native test a Yaci Store build, and --only / --ccl-version narrow the run - bun lockfiles only (the runner uses bun); node_modules, .venv and .logs are ignored All 13 tests pass against Yaci Store 2.0.4 (#193). Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

satran004 and others added 5 commits September 17, 2026 13:06

satran004 changed the base branch from feat/ogmios_kupo_update to main September 26, 2026 11:41

satran004 and others added 4 commits September 26, 2026 19:42

satran004 mentioned this pull request Sep 30, 2026

SDK compatibility e2e suites (Evolution SDK, Lucid, MeshJS, PyCardano, cardano-client-lib) and a runner #194

Merged

satran004 merged commit 1b5da80 into main Sep 30, 2026
2 checks passed

satran004 mentioned this pull request Oct 4, 2026

yano: pass a backfill block interval derived from the stability window #191

Closed

Navigation Menu

Repository navigation

Use Yano's automatic sparse backfill and KES defaults that outlive a devnet - #193

Use Yano's automatic sparse backfill and KES defaults that outlive a devnet#193
satran004 merged 9 commits into
mainfrom
feat/yano-backfill-kes-defaults

satran004 commented Sep 17, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

satran004 commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Automatic sparse backfill

KES defaults

Relationship to #191

A note for whoever integrates rjharmon's branch

Updates (2026-09-26)

Measurements (Yano pre16, same build, only yano.backfill.block.interval.slots changed)

Docker verification (native parity)

Merged with main (2026-09-26)

Yaci Store 2.0.4 (2026-09-30)

Test plan

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

satran004 commented Sep 17, 2026 •

edited

Loading

Measurements (Yano pre16, same build, only `yano.backfill.block.interval.slots` changed)

Merged with `main` (2026-09-26)