Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,12 +177,22 @@ Do not start the Rust backend until this workflow is usable end-to-end.
- [x] Burn policy/value update spike
- [x] Candle inference/checkpoint spike
- [x] Benchmark report
- [x] Backend decision report

### Success criteria

- Native engine can receive rollouts, train/update a policy and return actions/checkpoints.
- JS/R3F authoring API does not change because of backend choice.

### M5.1 decision

Build the next native slice around a backend-agnostic JSONL engine process and
custom Rust systems for orchestration, replay/update batches, checkpoint IO,
metrics and process supervision. Use Burn as the first training adapter behind
the Rust backend boundary and Candle as the first inference/checkpoint
portability adapter. Keep the final tensor-backend choice reversible until
Studio can run a real local training workflow end to end.

---

## M6 — Real MVP release
Expand Down
19 changes: 19 additions & 0 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,25 @@ For Studio ↔ Rust engine communication:

The SDK exports the protocol as `ENGINE_IPC_PROTOCOL_VERSION`, typed request/response/event envelopes and payloads for actions, transitions, metrics and checkpoints. `crates/ignitionrl-engine` mirrors the JSONL control plane before Burn or Candle-specific learner internals are selected.

## Native backend decision

The M5.1 backend decision is to keep Rust backend-agnostic at the Studio
boundary. The JSONL engine process is the control plane; custom Rust systems own
run orchestration, replay/update batches, checkpoint IO, metric streaming and
process supervision. Burn is the first training adapter behind the Rust
`EngineBackend` boundary. Candle is the first lightweight inference and
checkpoint portability adapter.

This decision is intentionally reversible. Burn and Candle must remain hidden
behind engine messages and backend adapters. Studio, SDK view models and
environment definitions should expose training capabilities, run state,
artifacts and diagnostics rather than backend-specific concepts. If larger
rollout payloads make JSONL too expensive, add a binary side channel while
keeping the JSON protocol as the stable control plane.

See [`RUST_BACKEND_DECISION.md`](./RUST_BACKEND_DECISION.md) for the comparison,
risks, rollback plan and decision log.

## Storage

Local-first project format:
Expand Down
261 changes: 146 additions & 115 deletions docs/RUST_BACKEND_DECISION.md
Original file line number Diff line number Diff line change
@@ -1,118 +1,110 @@
# Rust Backend Decision Notes

## Decision to make

IgnitionRL should eventually have a native Rust engine, but the first product validation should be TypeScript-first.

The backend decision should be based on experiments around PPO/DQN, serialization, checkpointing, CPU/GPU support, and integration with Tauri/Bun.

## Candidate: Burn

### Why it is attractive

- Training-first positioning.
- Rust-native deep learning.
- Dynamic graph/shapes with runtime optimization ambitions.
- Multi-device portability narrative.

### Risks

- Ecosystem maturity vs Python/PyTorch.
- Need to verify RL-specific ergonomics.
- Documentation and examples may not map directly to PPO/SAC style workloads.

### Spike

Implement a tiny MLP policy and value network for a toy PPO loop.

### Current result

`crates/ignitionrl-backend-burn` now validates Burn for a minimal native
# Rust Backend Decision Report

## Status

M5.1 decision, accepted for the next implementation slice.

This is a product and architecture decision, not a permanent tensor-framework
commitment. It can be revisited after Studio can run a real local training loop
against a saved project and at least one DQN or PPO path has been exercised end
to end.

## Decision

IgnitionRL should build the native engine around a backend-agnostic Rust control
plane first.

The next implementation should use:

- `ignitionrl-engine` as a long-running JSONL process owned by Studio;
- custom Rust systems for orchestration, replay/update batches, checkpoint IO,
metrics and process supervision;
- Burn as the first native training adapter behind the `EngineBackend`
boundary;
- Candle as the first lightweight inference and checkpoint portability adapter;
- the TypeScript environment contract as the stable user-facing authoring
boundary.

Burn is the strongest near-term candidate for training experiments. Candle is
the strongest near-term candidate for inference and checkpoint portability.
Neither should leak into environment definitions, Studio view models or public
TypeScript authoring APIs.

## What to build next

1. Keep `ignitionrl-engine` as the stable process boundary for Studio.
2. Add a backend adapter registry behind the Rust `EngineBackend` trait.
3. Route run lifecycle, transitions, metrics and checkpoints through backend
independent engine messages.
4. Wire Burn first for train/update batches because the policy/value update
spike already validates the autodiff and optimizer path.
5. Wire Candle as an inference/checkpoint adapter for lightweight policy
loading, action selection and export experiments.
6. Keep Studio and SDK consumers on the existing JSON protocol so the backend
can change without changing environment code.
7. Re-evaluate the primary tensor backend only after a real local Studio
training workflow exists and records comparable evidence for learning
quality, checkpoint reload, cancellation and packaging.

## What remains reversible

- Burn vs Candle as the primary tensor backend remains reversible behind
`EngineBackend`.
- Checkpoint payload internals remain reversible as long as metadata includes a
format version, backend id and schema version.
- JSONL remains the control plane; a binary side channel for large rollout
batches can be added later without replacing request, response and event
envelopes.
- Environment authoring remains TypeScript-first and must not depend on Rust
backend selection.
- Studio UI should expose training capability and run state, not Burn or Candle
implementation details.

## Candidate comparison

| Need | Burn | Candle | Custom Rust systems |
|---|---|---|---|
| Training | Best current candidate. The spike validates a tiny policy/value update with autodiff and SGD, but PPO/DQN ergonomics and long-running learner behavior still need proof. | Not validated for custom RL training yet. Treat as a riskier training choice until an update spike exists. | Not a tensor backend. Useful for learner orchestration, batching, replay and deterministic data movement. |
| Inference | Plausible, but not the strongest reason to choose Burn today. | Strong fit. The spike validates CPU policy inference and checkpoint load/save for small MLP policies. | Can host policy interfaces, action routing and process lifecycle, but should call a backend for tensor inference. |
| Checkpoints | Needs an IgnitionRL wrapper format around backend-specific state. | Strong fit for portable lightweight checkpoint experiments. | Should own checkpoint metadata, run linkage, versioning and filesystem IO independent of tensor backend. |
| Portability | Promising multi-device story, but packaging and platform behavior still need product evidence. | Good lightweight and WASM signal, plus Hugging Face ecosystem proximity. | Strongest control over local-first Studio packaging, JSON artifacts and future Tauri integration. |
| Studio integration | Should be hidden behind engine messages and backend adapters. | Should be hidden behind engine messages and backend adapters. | Best fit for Studio-facing process supervision, streaming status, cancellation, metrics and artifact persistence. |

## Evidence

### Burn spike

`crates/ignitionrl-backend-burn` validates Burn for a minimal native
policy/value update. The spike uses `burn-ndarray` on CPU with
`burn-autodiff`, a shared-hidden-layer MLP, a policy-gradient-style log-prob
loss over sampled actions and advantages, value-head MSE over returns, and an
SGD optimizer step through Burn's optimizer API.

This is a useful signal for PPO-style training ergonomics: Burn handles the
autodiff/optimizer/module path cleanly for a small RL-shaped update. It does not
yet validate the full learner runtime, rollout batching, checkpoint
serialization, or long-running Studio engine process.

## Candidate: Candle

### Why it is attractive

- Minimalist ML framework in Rust.
- Hugging Face ecosystem proximity.
- Strong inference story.
- WASM examples and lightweight deployment potential.
This is useful signal for PPO-style training ergonomics. It does not yet
validate a full learner runtime, rollout batching, checkpoint serialization,
GPU behavior or a long-running Studio engine process.

### Risks
### Candle spike

- Training ergonomics for custom RL loops must be validated.
- May be more inference-oriented than training-productivity-oriented.
`crates/ignitionrl-backend-candle` validates Candle for lightweight policy
inference and checkpoint loading/saving. The spike uses `candle-core` tensors on
CPU for linear or one-hidden-layer MLP logits and argmax action selection.

### Spike
This is a good fit for checkpoint portability and inference experiments. It
does not yet prove Candle training ergonomics for custom RL updates.

Implement policy inference, then test whether PPO update ergonomics are acceptable.
### Custom Rust systems

### Current result
`crates/ignitionrl-core` provides backend-agnostic transitions, rollouts, replay
buffer, metric and checkpoint data structures. `crates/ignitionrl-engine`
provides the JSONL process prototype for handshake, run lifecycle, action,
transition, metric, checkpoint, status, protocol-error and shutdown paths.

`crates/ignitionrl-backend-candle` now validates Candle for lightweight policy
inference and checkpoint loading/saving. The implemented spike uses
`candle-core` tensors on CPU for linear or one-hidden-layer MLP logits and argmax
action selection. This is a good fit for checkpoint portability and inference
experiments, but it does not yet prove Candle training ergonomics for custom RL
updates.
This gives Studio value before a final tensor backend choice: process
supervision, replay/update batching, checkpoint IO and streaming telemetry are
all useful even if the training backend changes later.

## Candidate: Rust for systems, not tensors

A pragmatic route is to keep neural training backend swappable and use Rust first for:

- rollout storage;
- replay buffer;
- vectorized environment stepping;
- serialization;
- compression;
- experiment storage;
- native process orchestration.

This gives value even before committing to a Rust tensor framework.

## Recommended sequence

1. TS-first contract and Studio UX.
2. Random/scripted policy and one simple learner.
3. Rust rollout/replay buffer crate.
4. Burn policy/value update spike.
5. Candle inference spike.
6. Define the backend abstraction and compare end-to-end engine integration.
7. Choose one primary native backend only after a real local Studio training loop.

## Decision matrix

| Criteria | Burn | Candle | Custom Rust systems |
|---|---:|---:|---:|
| Training ergonomics | Validated for a tiny policy/value update; PPO still pending | Not validated for training | N/A |
| Inference portability | Good potential | Strong potential | Depends |
| WASM potential | To validate | Good signal | Good for systems |
| Hugging Face proximity | Lower | Higher | N/A |
| Product velocity now | Medium | Medium | High |
| Long-term native engine | Strong candidate | Strong candidate | Strong foundation |

## Default recommendation

Do not choose permanently yet.

Start with a backend abstraction after the two backend spikes:

- Burn remains the better candidate for native training experiments.
- Candle remains the better candidate for lightweight inference/checkpoint portability experiments.

The next Rust work should be backend-agnostic engine infrastructure, not a
deeper neural net commitment.

## Engine benchmark snapshot
### Benchmark snapshot

Command:

Expand Down Expand Up @@ -147,20 +139,59 @@ Latest local run:

Interpretation:

- TS remains faster for a trivial in-process toy step, so Rust should not be
introduced just to move simple environment stepping out of JS.
- TypeScript remains faster for a trivial in-process toy step, so Rust should
not be introduced just to move simple environment stepping out of JS.
- Rust replay-buffer insertion and checkpoint serialization are cheap enough for
local project plumbing.
- JSONL per-action overhead is acceptable for a long-running local engine
control plane, but cold start means Studio should keep the engine process warm
during active training instead of spawning per action or per episode.
- Burn's tiny policy/value update remains promising for native learner spikes,
but this benchmark is still synthetic. It does not validate full PPO, GPU
behavior, R3F scene stepping or large rollout batches.

Current recommendation after the benchmark:

Keep the backend-agnostic JSONL process path. Use Rust first for run
orchestration, replay/update batches, checkpoint IO and metric streaming, while
keeping TypeScript as the authoring and simple simulation loop until Studio has a
real local-training workflow.
- Burn's tiny policy/value update is promising for native learner spikes, but
the benchmark is synthetic. It does not validate full PPO, GPU behavior, R3F
scene stepping or large rollout batches.

## Risks

- Burn may not provide the best ergonomics for full PPO, DQN replay updates,
checkpoint migration or packaged desktop distribution.
- Candle may remain more inference-oriented than training-productivity-oriented.
- JSONL overhead can become too high for large tensor payloads or high-frequency
per-step action calls if misused as a data plane.
- Native packaging can add build, signing and platform support complexity before
the product workflow is validated.
- The current benchmark is synthetic and may not predict real Studio runs with
R3F scenes, larger observations or long-running learner state.
- Studio process supervision must handle cancellation, partial artifacts,
malformed backend messages and engine crashes before user-facing training is
credible.

## Rollback plan

If Burn blocks training progress, keep the JSONL engine process and custom Rust
systems, then move training back to the TypeScript learner path or add a
different backend adapter behind `EngineBackend`.

If Candle fails to provide useful checkpoint portability, keep checkpoint
metadata and filesystem IO in custom Rust systems, then serialize backend-owned
state through the Burn adapter or a neutral JSON/binary payload format.

If JSONL becomes too expensive for rollout payloads, keep it as the versioned
control plane and add a binary side channel for batch data.

If native packaging slows product validation, gate native training behind an
experimental capability while keeping TypeScript headless learners, run
artifacts and Studio inspection workflows available.

In every rollback path, the TypeScript environment contract remains unchanged.

## Decision log

| Review role | Objection or constraint | Resolution |
|---|---|---|
| Primary designer | The product needs backend progress without freezing the wrong ML framework. | Choose a backend-agnostic Rust control plane and adapter boundary. |
| Skeptic | The Burn spike is too small to justify a permanent training decision. | Accepted. Burn is first training adapter, not a permanent platform choice. |
| Constraint guardian | JSONL is safe as a control plane but risky for large data payloads. | Accepted. Keep JSONL envelopes stable and reserve a binary side channel for batch data. |
| User advocate | Environment authors should not see backend churn. | Accepted. TypeScript authoring and Studio view models remain backend-neutral. |
| Integrator | The decision is acceptable only if the next build path and rollback plan are explicit. | Accepted. The report defines next work, reversible choices, risks and rollback paths. |

Final disposition: APPROVED for M5.1 implementation planning.
Loading