You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit e307910
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: packages/learning/README.md
+31-3Lines changed: 31 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,9 +2,10 @@
2
2
3
3
Baseline learning helpers for IgnitionRL.
4
4
5
-
This package intentionally starts small. It does not implement PPO, DQN, SAC, tensors or native backends. The first learners are deterministic baselines for validating environment loops, transitions, metrics and checkpoint plumbing:
5
+
This package intentionally starts small. It does not implement PPO, SAC, native tensor backends or high-throughput training. The first learners validate environment loops, transitions, metrics and checkpoint plumbing:
6
6
7
7
- tabular Q-learning for discrete environments;
8
+
- DQN for small deterministic discrete debug environments;
8
9
- linear policy search for bounded continuous environments.
`DqnLearner` is the first neural learner path. It is intentionally compact: a seeded one-hidden-layer MLP, replay buffer, epsilon decay and target network for discrete toy/debug environments. It is not the final high-throughput backend, but it proves that neural learners can train, emit stable metrics and checkpoint behind the same environment contract.
@@ -112,7 +140,7 @@ Checkpoints are JSON-serializable and include:
112
140
- action shape and bounds, or discrete action values;
113
141
- learner config;
114
142
- metrics;
115
-
- learned Q-table or policy weights.
143
+
- learned Q-table, DQN network weights or policy weights.
116
144
117
145
`linear-policy-search` keeps v1 checkpoint loading backward-compatible when newer diagnostic metrics are missing. Loading normalizes those fields before inference, so older demo artifacts can still be replayed through the current learner.
118
146
@@ -146,7 +174,7 @@ Built-in support profiles are intentionally conservative:
146
174
147
175
Unsupported combinations fail before a run starts with an algorithm-specific error. Custom neural adapters must declare their supported action spaces explicitly.
148
176
149
-
Current `TabularQLearner`and `LinearPolicySearchLearner` remain direct TypeScript learners. Future DQN/PPO/SAC implementations can sit behind this contract whether the backend is TypeScript, Rust/Burn, Rust/Candle or another native process. Environment authors still only implement `defineEnvironment()`.
177
+
Current `TabularQLearner`, `DqnLearner`and `LinearPolicySearchLearner` remain direct TypeScript learners. Future PPO/SAC implementations, and heavier DQN backends, can sit behind this contract whether the backend is TypeScript, Rust/Burn, Rust/Candle or another native process. Environment authors still only implement `defineEnvironment()`.
0 commit comments