From d5b03f58ff17f4fabd61ad35a4715e5b53225f25 Mon Sep 17 00:00:00 2001 From: salimlaimeche Date: Thu, 28 May 2026 22:11:36 +0200 Subject: [PATCH] docs(engine): publish backend decision report --- ROADMAP.md | 10 ++ docs/ARCHITECTURE.md | 19 +++ docs/RUST_BACKEND_DECISION.md | 261 +++++++++++++++++++--------------- 3 files changed, 175 insertions(+), 115 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 415c5e4..1b2088b 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -177,12 +177,22 @@ Do not start the Rust backend until this workflow is usable end-to-end. - [x] Burn policy/value update spike - [x] Candle inference/checkpoint spike - [x] Benchmark report +- [x] Backend decision report ### Success criteria - Native engine can receive rollouts, train/update a policy and return actions/checkpoints. - JS/R3F authoring API does not change because of backend choice. +### M5.1 decision + +Build the next native slice around a backend-agnostic JSONL engine process and +custom Rust systems for orchestration, replay/update batches, checkpoint IO, +metrics and process supervision. Use Burn as the first training adapter behind +the Rust backend boundary and Candle as the first inference/checkpoint +portability adapter. Keep the final tensor-backend choice reversible until +Studio can run a real local training workflow end to end. + --- ## M6 — Real MVP release diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index ed3e6c9..b44da58 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -229,6 +229,25 @@ For Studio ↔ Rust engine communication: The SDK exports the protocol as `ENGINE_IPC_PROTOCOL_VERSION`, typed request/response/event envelopes and payloads for actions, transitions, metrics and checkpoints. `crates/ignitionrl-engine` mirrors the JSONL control plane before Burn or Candle-specific learner internals are selected. +## Native backend decision + +The M5.1 backend decision is to keep Rust backend-agnostic at the Studio +boundary. The JSONL engine process is the control plane; custom Rust systems own +run orchestration, replay/update batches, checkpoint IO, metric streaming and +process supervision. Burn is the first training adapter behind the Rust +`EngineBackend` boundary. Candle is the first lightweight inference and +checkpoint portability adapter. + +This decision is intentionally reversible. Burn and Candle must remain hidden +behind engine messages and backend adapters. Studio, SDK view models and +environment definitions should expose training capabilities, run state, +artifacts and diagnostics rather than backend-specific concepts. If larger +rollout payloads make JSONL too expensive, add a binary side channel while +keeping the JSON protocol as the stable control plane. + +See [`RUST_BACKEND_DECISION.md`](./RUST_BACKEND_DECISION.md) for the comparison, +risks, rollback plan and decision log. + ## Storage Local-first project format: diff --git a/docs/RUST_BACKEND_DECISION.md b/docs/RUST_BACKEND_DECISION.md index 43a16b1..47a28de 100644 --- a/docs/RUST_BACKEND_DECISION.md +++ b/docs/RUST_BACKEND_DECISION.md @@ -1,118 +1,110 @@ -# Rust Backend Decision Notes - -## Decision to make - -IgnitionRL should eventually have a native Rust engine, but the first product validation should be TypeScript-first. - -The backend decision should be based on experiments around PPO/DQN, serialization, checkpointing, CPU/GPU support, and integration with Tauri/Bun. - -## Candidate: Burn - -### Why it is attractive - -- Training-first positioning. -- Rust-native deep learning. -- Dynamic graph/shapes with runtime optimization ambitions. -- Multi-device portability narrative. - -### Risks - -- Ecosystem maturity vs Python/PyTorch. -- Need to verify RL-specific ergonomics. -- Documentation and examples may not map directly to PPO/SAC style workloads. - -### Spike - -Implement a tiny MLP policy and value network for a toy PPO loop. - -### Current result - -`crates/ignitionrl-backend-burn` now validates Burn for a minimal native +# Rust Backend Decision Report + +## Status + +M5.1 decision, accepted for the next implementation slice. + +This is a product and architecture decision, not a permanent tensor-framework +commitment. It can be revisited after Studio can run a real local training loop +against a saved project and at least one DQN or PPO path has been exercised end +to end. + +## Decision + +IgnitionRL should build the native engine around a backend-agnostic Rust control +plane first. + +The next implementation should use: + +- `ignitionrl-engine` as a long-running JSONL process owned by Studio; +- custom Rust systems for orchestration, replay/update batches, checkpoint IO, + metrics and process supervision; +- Burn as the first native training adapter behind the `EngineBackend` + boundary; +- Candle as the first lightweight inference and checkpoint portability adapter; +- the TypeScript environment contract as the stable user-facing authoring + boundary. + +Burn is the strongest near-term candidate for training experiments. Candle is +the strongest near-term candidate for inference and checkpoint portability. +Neither should leak into environment definitions, Studio view models or public +TypeScript authoring APIs. + +## What to build next + +1. Keep `ignitionrl-engine` as the stable process boundary for Studio. +2. Add a backend adapter registry behind the Rust `EngineBackend` trait. +3. Route run lifecycle, transitions, metrics and checkpoints through backend + independent engine messages. +4. Wire Burn first for train/update batches because the policy/value update + spike already validates the autodiff and optimizer path. +5. Wire Candle as an inference/checkpoint adapter for lightweight policy + loading, action selection and export experiments. +6. Keep Studio and SDK consumers on the existing JSON protocol so the backend + can change without changing environment code. +7. Re-evaluate the primary tensor backend only after a real local Studio + training workflow exists and records comparable evidence for learning + quality, checkpoint reload, cancellation and packaging. + +## What remains reversible + +- Burn vs Candle as the primary tensor backend remains reversible behind + `EngineBackend`. +- Checkpoint payload internals remain reversible as long as metadata includes a + format version, backend id and schema version. +- JSONL remains the control plane; a binary side channel for large rollout + batches can be added later without replacing request, response and event + envelopes. +- Environment authoring remains TypeScript-first and must not depend on Rust + backend selection. +- Studio UI should expose training capability and run state, not Burn or Candle + implementation details. + +## Candidate comparison + +| Need | Burn | Candle | Custom Rust systems | +|---|---|---|---| +| Training | Best current candidate. The spike validates a tiny policy/value update with autodiff and SGD, but PPO/DQN ergonomics and long-running learner behavior still need proof. | Not validated for custom RL training yet. Treat as a riskier training choice until an update spike exists. | Not a tensor backend. Useful for learner orchestration, batching, replay and deterministic data movement. | +| Inference | Plausible, but not the strongest reason to choose Burn today. | Strong fit. The spike validates CPU policy inference and checkpoint load/save for small MLP policies. | Can host policy interfaces, action routing and process lifecycle, but should call a backend for tensor inference. | +| Checkpoints | Needs an IgnitionRL wrapper format around backend-specific state. | Strong fit for portable lightweight checkpoint experiments. | Should own checkpoint metadata, run linkage, versioning and filesystem IO independent of tensor backend. | +| Portability | Promising multi-device story, but packaging and platform behavior still need product evidence. | Good lightweight and WASM signal, plus Hugging Face ecosystem proximity. | Strongest control over local-first Studio packaging, JSON artifacts and future Tauri integration. | +| Studio integration | Should be hidden behind engine messages and backend adapters. | Should be hidden behind engine messages and backend adapters. | Best fit for Studio-facing process supervision, streaming status, cancellation, metrics and artifact persistence. | + +## Evidence + +### Burn spike + +`crates/ignitionrl-backend-burn` validates Burn for a minimal native policy/value update. The spike uses `burn-ndarray` on CPU with `burn-autodiff`, a shared-hidden-layer MLP, a policy-gradient-style log-prob loss over sampled actions and advantages, value-head MSE over returns, and an SGD optimizer step through Burn's optimizer API. -This is a useful signal for PPO-style training ergonomics: Burn handles the -autodiff/optimizer/module path cleanly for a small RL-shaped update. It does not -yet validate the full learner runtime, rollout batching, checkpoint -serialization, or long-running Studio engine process. - -## Candidate: Candle - -### Why it is attractive - -- Minimalist ML framework in Rust. -- Hugging Face ecosystem proximity. -- Strong inference story. -- WASM examples and lightweight deployment potential. +This is useful signal for PPO-style training ergonomics. It does not yet +validate a full learner runtime, rollout batching, checkpoint serialization, +GPU behavior or a long-running Studio engine process. -### Risks +### Candle spike -- Training ergonomics for custom RL loops must be validated. -- May be more inference-oriented than training-productivity-oriented. +`crates/ignitionrl-backend-candle` validates Candle for lightweight policy +inference and checkpoint loading/saving. The spike uses `candle-core` tensors on +CPU for linear or one-hidden-layer MLP logits and argmax action selection. -### Spike +This is a good fit for checkpoint portability and inference experiments. It +does not yet prove Candle training ergonomics for custom RL updates. -Implement policy inference, then test whether PPO update ergonomics are acceptable. +### Custom Rust systems -### Current result +`crates/ignitionrl-core` provides backend-agnostic transitions, rollouts, replay +buffer, metric and checkpoint data structures. `crates/ignitionrl-engine` +provides the JSONL process prototype for handshake, run lifecycle, action, +transition, metric, checkpoint, status, protocol-error and shutdown paths. -`crates/ignitionrl-backend-candle` now validates Candle for lightweight policy -inference and checkpoint loading/saving. The implemented spike uses -`candle-core` tensors on CPU for linear or one-hidden-layer MLP logits and argmax -action selection. This is a good fit for checkpoint portability and inference -experiments, but it does not yet prove Candle training ergonomics for custom RL -updates. +This gives Studio value before a final tensor backend choice: process +supervision, replay/update batching, checkpoint IO and streaming telemetry are +all useful even if the training backend changes later. -## Candidate: Rust for systems, not tensors - -A pragmatic route is to keep neural training backend swappable and use Rust first for: - -- rollout storage; -- replay buffer; -- vectorized environment stepping; -- serialization; -- compression; -- experiment storage; -- native process orchestration. - -This gives value even before committing to a Rust tensor framework. - -## Recommended sequence - -1. TS-first contract and Studio UX. -2. Random/scripted policy and one simple learner. -3. Rust rollout/replay buffer crate. -4. Burn policy/value update spike. -5. Candle inference spike. -6. Define the backend abstraction and compare end-to-end engine integration. -7. Choose one primary native backend only after a real local Studio training loop. - -## Decision matrix - -| Criteria | Burn | Candle | Custom Rust systems | -|---|---:|---:|---:| -| Training ergonomics | Validated for a tiny policy/value update; PPO still pending | Not validated for training | N/A | -| Inference portability | Good potential | Strong potential | Depends | -| WASM potential | To validate | Good signal | Good for systems | -| Hugging Face proximity | Lower | Higher | N/A | -| Product velocity now | Medium | Medium | High | -| Long-term native engine | Strong candidate | Strong candidate | Strong foundation | - -## Default recommendation - -Do not choose permanently yet. - -Start with a backend abstraction after the two backend spikes: - -- Burn remains the better candidate for native training experiments. -- Candle remains the better candidate for lightweight inference/checkpoint portability experiments. - -The next Rust work should be backend-agnostic engine infrastructure, not a -deeper neural net commitment. - -## Engine benchmark snapshot +### Benchmark snapshot Command: @@ -147,20 +139,59 @@ Latest local run: Interpretation: -- TS remains faster for a trivial in-process toy step, so Rust should not be - introduced just to move simple environment stepping out of JS. +- TypeScript remains faster for a trivial in-process toy step, so Rust should + not be introduced just to move simple environment stepping out of JS. - Rust replay-buffer insertion and checkpoint serialization are cheap enough for local project plumbing. - JSONL per-action overhead is acceptable for a long-running local engine control plane, but cold start means Studio should keep the engine process warm during active training instead of spawning per action or per episode. -- Burn's tiny policy/value update remains promising for native learner spikes, - but this benchmark is still synthetic. It does not validate full PPO, GPU - behavior, R3F scene stepping or large rollout batches. - -Current recommendation after the benchmark: - -Keep the backend-agnostic JSONL process path. Use Rust first for run -orchestration, replay/update batches, checkpoint IO and metric streaming, while -keeping TypeScript as the authoring and simple simulation loop until Studio has a -real local-training workflow. +- Burn's tiny policy/value update is promising for native learner spikes, but + the benchmark is synthetic. It does not validate full PPO, GPU behavior, R3F + scene stepping or large rollout batches. + +## Risks + +- Burn may not provide the best ergonomics for full PPO, DQN replay updates, + checkpoint migration or packaged desktop distribution. +- Candle may remain more inference-oriented than training-productivity-oriented. +- JSONL overhead can become too high for large tensor payloads or high-frequency + per-step action calls if misused as a data plane. +- Native packaging can add build, signing and platform support complexity before + the product workflow is validated. +- The current benchmark is synthetic and may not predict real Studio runs with + R3F scenes, larger observations or long-running learner state. +- Studio process supervision must handle cancellation, partial artifacts, + malformed backend messages and engine crashes before user-facing training is + credible. + +## Rollback plan + +If Burn blocks training progress, keep the JSONL engine process and custom Rust +systems, then move training back to the TypeScript learner path or add a +different backend adapter behind `EngineBackend`. + +If Candle fails to provide useful checkpoint portability, keep checkpoint +metadata and filesystem IO in custom Rust systems, then serialize backend-owned +state through the Burn adapter or a neutral JSON/binary payload format. + +If JSONL becomes too expensive for rollout payloads, keep it as the versioned +control plane and add a binary side channel for batch data. + +If native packaging slows product validation, gate native training behind an +experimental capability while keeping TypeScript headless learners, run +artifacts and Studio inspection workflows available. + +In every rollback path, the TypeScript environment contract remains unchanged. + +## Decision log + +| Review role | Objection or constraint | Resolution | +|---|---|---| +| Primary designer | The product needs backend progress without freezing the wrong ML framework. | Choose a backend-agnostic Rust control plane and adapter boundary. | +| Skeptic | The Burn spike is too small to justify a permanent training decision. | Accepted. Burn is first training adapter, not a permanent platform choice. | +| Constraint guardian | JSONL is safe as a control plane but risky for large data payloads. | Accepted. Keep JSONL envelopes stable and reserve a binary side channel for batch data. | +| User advocate | Environment authors should not see backend churn. | Accepted. TypeScript authoring and Studio view models remain backend-neutral. | +| Integrator | The decision is acceptable only if the next build path and rollback plan are explicit. | Accepted. The report defines next work, reversible choices, risks and rollback paths. | + +Final disposition: APPROVED for M5.1 implementation planning.