You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 42e02bb
Browse filesBrowse the repository at this point in the historyBrowse files
IgnitionRL needs DQN, PPO and SAC later, but environment authors should not see algorithm-specific internals. The existing public boundary is already the TypeScript environment contract: vector observations, action specs, rewards, done conditions, traces and checkpoints.
10
+
11
+
M3.1 needs an adapter contract before implementing neural algorithms so Studio, CI and future native backends can agree on action-space support, update cadence, metrics and checkpoint shape.
12
+
13
+
## Decision
14
+
15
+
Add a `@ignitionrl/learning` neural adapter contract that is derived from `EnvironmentSpec`:
16
+
17
+
-`NeuralObservationSpace` mirrors vector observation shape and dtype.
18
+
-`NeuralActionSpace` normalizes discrete, continuous and multi-discrete action specs.
19
+
-`NeuralUpdateCadence` models step-based updates, episode-based updates and rollout-based updates.
20
+
-`NeuralMetricSpec` defines stable metric names, scopes, reducers and optimization direction.
21
+
-`NeuralCheckpointEnvelope` wraps backend-specific checkpoint payloads in a stable JSON envelope.
22
+
23
+
The built-in algorithm support profiles are:
24
+
25
+
- DQN: discrete action spaces.
26
+
- PPO: discrete, continuous and multi-discrete action spaces.
27
+
- SAC: continuous action spaces.
28
+
29
+
Unsupported action spaces fail before training starts. Custom adapters must declare supported action spaces explicitly.
30
+
31
+
## Consequences
32
+
33
+
Future DQN, PPO and SAC implementations can plug in without changing `defineEnvironment()`.
34
+
35
+
Studio can reason about learner metrics and checkpoint payloads before the actual neural backend exists.
36
+
37
+
Native backends can keep tensor payloads opaque while still producing stable IgnitionRL metadata around observations, actions, update cadence and metrics.
38
+
39
+
The current tabular Q-learning and linear policy search baselines remain unchanged; they map conceptually to the same boundaries but are not forced through the neural adapter until a concrete neural implementation needs it.
Copy file name to clipboardExpand all lines: packages/learning/README.md
+32Lines changed: 32 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -81,6 +81,38 @@ Checkpoints are JSON-serializable and include:
81
81
82
82
`linear-policy-search` keeps v1 checkpoint loading backward-compatible when newer diagnostic metrics are missing. Loading normalizes those fields before inference, so older demo artifacts can still be replayed through the current learner.
83
83
84
+
## Neural Learner Adapter Contract
85
+
86
+
`@ignitionrl/learning` now exposes an adapter contract for future neural learners without changing environment definitions:
- discrete, continuous or multi-discrete action space details;
101
+
- algorithm support rules for DQN, PPO and SAC;
102
+
- update cadence (`step`, `episode` or `rollout`);
103
+
- stable metric names for Studio and CI;
104
+
- a JSON checkpoint envelope with an opaque backend payload.
105
+
106
+
Built-in support profiles are intentionally conservative:
107
+
108
+
-`dqn`: discrete action spaces;
109
+
-`ppo`: discrete, continuous and multi-discrete action spaces;
110
+
-`sac`: continuous action spaces.
111
+
112
+
Unsupported combinations fail before a run starts with an algorithm-specific error. Custom neural adapters must declare their supported action spaces explicitly.
113
+
114
+
Current `TabularQLearner` and `LinearPolicySearchLearner` remain direct TypeScript learners. Future DQN/PPO/SAC implementations can sit behind this contract whether the backend is TypeScript, Rust/Burn, Rust/Candle or another native process. Environment authors still only implement `defineEnvironment()`.
115
+
84
116
## Scope
85
117
86
118
Use this package to prove that environments can be trained and reloaded through the public contract. Neural RL algorithms and native training backends should come later behind the same `Learner` and checkpoint boundary.
0 commit comments