Normalize the extension surface: user-supplied model services, Promoter, enforced defaults

AGENTS.md

@@ -0,0 +1,52 @@
+# Rules for agents working in this repository
+
+These are maintainer decisions, not suggestions. Violating one is a hard
+failure: stop, revert the violating change, and re-read this file. Several
+have already been violated by agents "simplifying" the API; the enforcement
+tests named below exist because of those incidents.
+
+## Trainable identity (ADR — hard failure)
+
+**A string literal must never define a trainable identity in application
+code, examples, or documentation. Not as `trainable: "X"`, and not as
+`defineTrainable("X")`. Both are rejected.**
+
+The intended design, in full:
+
+1. The application declares a **`unique symbol`** it owns:
+
+   ```ts
+   export const route: unique symbol = Symbol("route");
+   ```
+
+2. The `@trainable(symbol)` **decorator on the trainable code** uses that
+   symbol as the key and registers the declaration under it.
+
+3. Training reuses the same symbol — `training.train(route)` — so discovery
+   is simple symbol-key indexing. Object identity of the symbol is the
+   uniqueness guarantee; no registry string, no retyped name, nothing a typo
+   can silently fork.
+
+What follows from this:
+
+- `Symbol.for(...)` registry symbols are a compatibility path for the
+  zero-config directive flow, where the *machinery* derives ids from parsed
+  source. Machinery-derived ids are allowed; user-typed ones are not.
+- `defineTrainable(id)` is machinery: source discovery, replay, and the
+  register hook use it internally. It must not appear in any README,
+  docs snippet, example, or suggested CLI output as something an
+  application calls.
+- Enforcement: `test/adr.test.ts` pins the string form as a compile error
+  and scans every documentation snippet for `defineTrainable(` — a doc that
+  teaches the banned pattern fails CI.
+
+## Other standing rules
+
+- Additive changes only within a release; deprecated aliases keep working
+  byte-for-byte (`test/deprecated.test.ts`).
+- Every injected seam ships a default (`test/defaults.test.ts`) and a
+  conformance suite; naming and shape conventions live in `CONTRIBUTING.md`.
+- Doc snippets compile in CI (`test/docs.test.ts`); a new doc must be added
+  to its list or its snippets silently go unchecked.
+- Root tests and docs resolve sibling packages through their built `dist/`;
+  rebuild the sibling before concluding a new type "doesn't exist".

CLAUDE.md

@@ -0,0 +1,52 @@
+# Rules for agents working in this repository
+
+These are maintainer decisions, not suggestions. Violating one is a hard
+failure: stop, revert the violating change, and re-read this file. Several
+have already been violated by agents "simplifying" the API; the enforcement
+tests named below exist because of those incidents.
+
+## Trainable identity (ADR — hard failure)
+
+**A string literal must never define a trainable identity in application
+code, examples, or documentation. Not as `trainable: "X"`, and not as
+`defineTrainable("X")`. Both are rejected.**
+
+The intended design, in full:
+
+1. The application declares a **`unique symbol`** it owns:
+
+   ```ts
+   export const route: unique symbol = Symbol("route");
+   ```
+
+2. The `@trainable(symbol)` **decorator on the trainable code** uses that
+   symbol as the key and registers the declaration under it.
+
+3. Training reuses the same symbol — `training.train(route)` — so discovery
+   is simple symbol-key indexing. Object identity of the symbol is the
+   uniqueness guarantee; no registry string, no retyped name, nothing a typo
+   can silently fork.
+
+What follows from this:
+
+- `Symbol.for(...)` registry symbols are a compatibility path for the
+  zero-config directive flow, where the *machinery* derives ids from parsed
+  source. Machinery-derived ids are allowed; user-typed ones are not.
+- `defineTrainable(id)` is machinery: source discovery, replay, and the
+  register hook use it internally. It must not appear in any README,
+  docs snippet, example, or suggested CLI output as something an
+  application calls.
+- Enforcement: `test/adr.test.ts` pins the string form as a compile error
+  and scans every documentation snippet for `defineTrainable(` — a doc that
+  teaches the banned pattern fails CI.
+
+## Other standing rules
+
+- Additive changes only within a release; deprecated aliases keep working
+  byte-for-byte (`test/deprecated.test.ts`).
+- Every injected seam ships a default (`test/defaults.test.ts`) and a
+  conformance suite; naming and shape conventions live in `CONTRIBUTING.md`.
+- Doc snippets compile in CI (`test/docs.test.ts`); a new doc must be added
+  to its list or its snippets silently go unchecked.
+- Root tests and docs resolve sibling packages through their built `dist/`;
+  rebuild the sibling before concluding a new type "doesn't exist".

CONTRIBUTING.md

@@ -19,5 +19,31 @@ npm run check
   behavior is not obvious.
 - Do not commit credentials, generated `dist/` files, or `.env` files.
 
+## Naming and shape conventions
+
+- Factories are `create*`; identity/value constructors are `define*`; nothing
+  else mints new verbs for construction.
+- Configuration objects: `…Settings` configures something long-lived (a
+  runtime, a service); `…Options` parameterizes one call or one factory
+  invocation. Do not introduce `…Config`/`…Params` variants.
+- Injected seams are agent nouns: `TrainingEngine`, `ImplementationExecutor`,
+  `TrainingLoop`, `TrainingStore`, `Promoter`, `PromotionGate`. Every seam
+  ships a default implementation (`test/defaults.test.ts` enforces it) and a
+  conformance suite (`packages/training/src/conformance.ts`); adding a seam
+  means adding both.
+- Provider choices are settings, not code: anything a user picks (a model, a
+  timeout, a threshold) belongs on `TrainingSettings`/`TrainInput`, carried
+  opaquely if provider-specific — never a hardcoded registry in this repo.
+- **A trainable identity is never a plain string (ADR).** A string is not a
+  sufficient identity to guarantee uniqueness. The design: the application
+  declares a `unique symbol`, `@trainable(symbol)` keys the code with it, and
+  `train(symbol)` is symbol-key indexing — the symbol's object identity is the
+  guarantee. The marked method itself also works (instrumentation stamps it).
+  String-typed identity, including `defineTrainable("...")` in examples, is a
+  hard failure: `test/adr.test.ts` pins the string form as a compile error and
+  scans every doc snippet for the banned pattern. See `AGENTS.md`.
+- Validate settings at the boundary with `parseSetting`, so misconfiguration
+  fails as `InvalidSettingsError` naming the setting, not deep in a run.
+
 The CI workflow runs type checking, tests, and the package build on supported
 Node.js versions.

README.md

@@ -46,9 +46,10 @@ Router.route  src/router.ts
 
 1 trainable.
 
-Bind evals to one with its symbol:
-  const target = defineTrainable("Router.route");
-  await training.train({ trainable: target.symbol, /* ... */ });
+Bind evals with your own symbol key:
+  export const route: unique symbol = Symbol("route");
+  // @trainable(route) on Router.route, then:
+  await training.train(route, { /* ... */ });
 ```
 
 `ts-autocode status` reports how many traces each trainable has captured, which
@@ -102,82 +103,63 @@ class Router {
 }
 ```
 
-When no symbol is passed, one is auto-generated for the decorated method: the
-identity above is `Router.route`, and `defineTrainable("Router.route").symbol`
-recreates its stable symbol anywhere. Pass a symbol explicitly for a durable
-id detached from the class name:
+Declare your own `unique symbol` and hand it to the decorator — the symbol is
+the key. `@trainable(route)` registers the method under it, and every training
+API reuses the *same* symbol, so discovery is plain symbol-key indexing and
+the symbol's object identity is the uniqueness guarantee. The durable string
+id the machinery needs (for stores and source rewriting) is derived from the
+declaring class and method — you never type a name anywhere:
 
 ```ts
-import { defineTrainable, trainable } from "ts-autocode";
+import { trainable } from "ts-autocode";
 
-const route = defineTrainable("acme.route");
+export const route: unique symbol = Symbol("route");
 
 class Router {
-  @trainable(route.symbol)
+  @trainable(route)
   route(input: string): string {
     return input;
   }
 }
 ```
 
-A token contains a durable string id and stable `Symbol.for(...)` symbol. The
-same symbol binds the method, its captures, AgentV results, optimizer
-candidate, and promotion decision — so evals, tests, and training reuse it to
-target exactly this trainable, binding evals to a training target at test time
-instead of only iterating during runtime.
+The same symbol binds the method, its captures, AgentV results, optimizer
+candidate, and promotion decision — evals, tests, and training all key off it
+to target exactly this trainable. The binding registers at first construction
+of the class.
 
 ## Train and activate
 
 AgentV owns eval definitions, graders, traces, scores, and result types. The
 `training` export is ready to use without any setup call.
 
-`train` takes the trainable's symbol (or its full token), never a raw string.
-Reusing `route.symbol` — the same symbol passed to `@trainable(route.symbol)`
-above — pins these evals to that exact method; for an auto-generated identity,
-`defineTrainable("Router.route").symbol` recreates the symbol.
+`train` takes the same symbol `@trainable(route)` put on the code — never a
+raw string. Symbol in the decorator, symbol at the call: one key, indexed.
 
 ```ts
-import { defineTrainable, training, type CandidatePatch } from "ts-autocode";
-
-// The same identity the marked method carries; see the section above.
-const route = defineTrainable("Router.route");
+import { trainable, training } from "ts-autocode";
 
-// Whatever your deployment rules are — anything returning a boolean works.
-const deploymentPolicy = {
-  allows: (candidate: CandidatePatch) => candidate.implementation.length < 4_000,
-};
+export const route: unique symbol = Symbol("route");
 
 class Router {
+  @trainable(route)
   route(input: string): string {
     "use training";
     return input.includes("invoice") ? "billing" : "fallback";
   }
 }
+void new Router();
 
-const router = new Router();
-
-const tests = [
-  {
-    id: "billing",
-    input: "Where is my invoice?",
-    assert: [{ type: "equals", value: "billing" }],
-  },
-  {
-    id: "fallback",
-    input: "Reset my password",
-    assert: [{ type: "equals", value: "fallback" }],
-  },
-];
-
-const run = await training.train({
-  trainable: route.symbol,
+const run = await training.train(route, {
   objective: "Preserve correct billing and fallback routing",
-  evaluation: {
-    tests,
-    task: (input) => router.route(input),
-    workers: 2,
+  cases: [
+    ["Where is my invoice?", "billing"],
+    ["Reset my password", "fallback"],
+  ],
+  promotion: {
+    // Any extra rule a candidate must clear, on top of the standard gates.
+    gates: [({ candidate }) => candidate.implementation.length < 4_000 ? undefined : "candidate too large"],
   },
-  policy: (candidate) => deploymentPolicy.allows(candidate),
 });
 
 const activation = await run.activate();
@@ -186,6 +168,24 @@ const activation = await run.activate();
 await activation.rollback();
 ```
 
+The identity is never a plain string — that is an ADR, enforced at compile
+time: a string is not a sufficient identity to guarantee uniqueness. The key
+is the symbol you declared (as above); the marked method itself also works,
+since instrumentation stamps it with the identity it registered:
+
+```ts
+import { training } from "ts-autocode";
+
+declare class Router { route(input: string): string; }
+
+await training.train(Router.prototype.route, { cases: [["a", "a"]] });
+```
+
+`cases` are `[input, expected]` pairs that become equality eval cases,
+evaluated exactly as replayed live traffic is. `evaluation.tests` with
+explicit asserts and a `task` remains the escape hatch when a case is not
+input/expected equality.
+
 Activating a training run writes the gated source rewrite and, for async
 targets, hot-swaps the running implementation through `ts-autocode-rewrite`'s
 AspectJS advice — woven methods dispatch to the promoted candidate immediately,
@@ -229,12 +229,13 @@ replacement, verifies the candidate against the same cases, and applies the
 promotion gate. Activating the run then updates the marked TypeScript body.
 
 ```ts
-import { defineTrainable, training } from "ts-autocode";
+import { training } from "ts-autocode";
 
-const route = defineTrainable("Router.route");
+// Under `--import ts-autocode/register`, the instrumented method itself is
+// an identity: it carries the id the machinery derived from the source.
+declare const router: { route(input: string): string };
 
-const run = await training.train({
-  trainable: route,
+const run = await training.train(router.route, {
   objective: "Preserve routing behavior observed in production",
   minTraces: 20,
   evaluation: {
@@ -398,8 +399,25 @@ secret provider, then the environment variable conventional for that provider
 (`ANTHROPIC_API_KEY`, `GOOGLE_API_KEY`, and so on). With nothing configured the
 default is OpenAI reading `OPENAI_API_KEY`.
 
-For Ax-specific tuning beyond model choice — a prepared `AxAIService`, or
-optimizer options — the `ts-autocode/ax` adapter builds an engine you pass
+The descriptor is sugar, not a support matrix. When it does not fit — a
+self-hosted endpoint, a proxy with its own auth, a client you have already
+built — supply the client itself as `model.service` and the library holds no
+opinion about providers at all. The default Ax engine accepts any
+`AxAIService`, or a factory returning one:
+
+```ts
+import { configureTraining } from "ts-autocode";
+import { ai } from "@ax-llm/ax";
+
+configureTraining({
+  model: {
+    service: ai({ name: "openai", apiKey: process.env.MY_PROXY_KEY ?? "", apiURL: "https://llm.internal.example" }),
+  },
+});
+```
+
+For Ax-specific tuning beyond model choice — optimizer options, a separate
+teacher service — the `ts-autocode/ax` adapter builds an engine you pass
 through the provider-neutral `engine` slot:
 
 ```ts

docs/architecture.md

@@ -5,10 +5,12 @@
 `TrainableToken` is the durable join key. A token id binds a trainable method to
 its runtime captures, AgentV evaluations, optimizer request, candidate, and
 promotion decision. Training APIs (`train`, `evaluate`, `records`) identify
-their target by the token or its symbol, never by a raw
-string; the `@trainable()` decorator auto-generates a token when no symbol is
-passed, and `defineTrainable(id)` recreates the same token and symbol anywhere,
-so tests and evals can bind to a training target directly.
+their target by the token or its symbol, never by a raw string. The
+application declares a `unique symbol` and puts it on the code with
+`@trainable(symbol)`; training reuses the same symbol, so discovery is plain
+symbol-key indexing, and the durable id is derived from the declaring class
+and method rather than typed anywhere. Internally the machinery mints tokens
+from ids it parses out of source; those ids never pass through user code.
 
 The first-statement `"use training"` directive is the consumer-facing marker.
 The TypeScript compiler API discovers the enclosing method directly and records
@@ -184,7 +186,8 @@ Every failure the library raises carries a `code` and is recognized by
 `SyntaxError`s still are — family membership is decided by a brand rather than
 the prototype chain — and every message string is unchanged.
 
-`ts-autocode discover` lists the trainables a project marks along with the exact
-identity to pass to `defineTrainable`. That identity is the one place this
-design falls back to an unchecked string, so printing real ids is what keeps the
-marker approach usable.
+`ts-autocode discover` lists the trainables a project marks with their derived
+ids, and its suggested snippet shows the symbol-key flow: declare a
+`unique symbol`, decorate the printed method with `@trainable(symbol)`, and
+train with the same symbol. The printed id is informational — it names which
+method to decorate, and is never something an application types back in.