flybrain: make a brain usable -- carry it forward, ship it with a bui… - #98
Merged
Merged
Conversation
…ld, verify it Five things a user hits, and the measurement tool that would have caught them. **pollard-brainverify.** The one measurement, with the construction that produced the published numbers built in. Rebuilding it by hand cost an evening and three different wrong answers -- 45.8%, 33.3%, 56.2% -- on a brain that measures 100%. It reports the floor, the recall, and BOTH leakage controls, and exits non-zero if any control is above 2%, because a number without its controls is not a measurement. **The query default was wrong, and so was my first correction.** A token is filed under the words immediately before it, so retrieval reproduces that context. The verified prompt is "Question: what is the secret word? Answer: The secret word is" -- question AND continuation. Shipping only the question returns confident noise from a perfect memory; shipping only the continuation is wrong the same way. Both halves now, and a test pins it against the trainer's own ASKS and the verifier's construction. **recall() crashed on most documents.** A final window shorter than `ek` left a short carry, and the next address slice died on a shape mismatch -- on any document whose length does not divide the window, which is nearly all of them. **--continue-from.** Training could only start over. On the same backbone this is cumulative curriculum -- run 900 steps, look at it, run 900 more. On a DIFFERENT backbone the memory-shaped parts (connectome, slots, codebook, decay) carry over and only the hidden-size-shaped layers are re-initialised, so a brain MOVES to a new model instead of starting from nothing; what is re-init reconverges in tens of steps because the memory it is learning to address is already organised. Changing bits/span between runs is refused outright -- that silently re-rolls the codebook and everything already written decodes to noise. **--codec bytes.** The payload was this backbone's vocabulary indices, so only that tokenizer could read a brain back. Bytes store TEXT: any tokenizer can read it, and a six-letter word costs 6x8=48 bits against 4x18=72. The codec is recorded in the checkpoint and defaults to the original behaviour, because reading a brain with the wrong codec is silent garbage. **pollard --brain.** A brain now ships WITH a build on every lane, with a BRAIN.md saying what it attaches to. GPTQ loads under transformers so it attaches at run time; GGUF, MLX and EXL3 run under engines with no hook for injecting memory tokens, so the file travels with the build and the note says so rather than leaving someone to discover it. Also: training progress was invisible when redirected (block-buffered stdout, so `--train > log` wrote nothing and read as a hung job), and the SKILL state-size line was stale at 8.8 MB after the width default moved to 328. 48/48 pass.
…n a quantized model Three changes, all opt-in: a brain written before them loads and behaves exactly as it did, which the verified 100% brain confirms at 32 samples with every control at 0%. **--canon: one brain, any hidden size.** A brain's address matrix was sized by the backbone, so it was locked to one model. With --canon it trains in a fixed canonical width and bridges to whatever it is attached to, using a probe basis built at attach time from the backbone's OWN responses to fixed probe texts -- so direction i means the same thing on every model, measured at +0.53 correspondence across families against -0.02 for a random per-model projection. Nothing about the bridge is trained. The return path is a pseudo-inverse, not a transpose: a transpose measured a round-trip gain of 7.5 and the brain could not learn through it at all. **A payload change no longer aborts a transfer.** --continue-from carries trained WEIGHTS; written memory lives in a .flystate file, so a new codebook has nothing stored to corrupt. Changing bits, span or codec resizes the value and decoder layers and re-initialises exactly those, while the address path, gate and connectome carry across. People swap backbones and payloads constantly. A .flystate from a differently shaped brain IS still refused, in load_state, where real memory lives. **pollard_brain_backends: the runtime seam.** A brain needs four operations from a backbone -- embed(ids), forward(embeds) -> (logits, hidden), forward_ids(ids), out_weight() -- and nothing else. Brains ran only under transformers because those calls were written inline against one library, not because of the memory, the connectome or the payload. Transformers, MLX, exllamav3 and llama.cpp are implementations of that protocol now. MLX is verified working end to end on a 4-BIT QUANTIZED model: backend mlx, hidden 896 embed (1, 6, 896) forward logits (1, 6, 151936) hidden (1, 6, 896) out_weight (151936, 896) That last line was a bug worth the whole exercise: a quantized MLX model reports its output embedding PACKED -- (151936, 112) for 4-bit -- and the brain derives its token codes from it. Handed packed bytes it builds a codebook out of bit-patterns and every stored token decodes to noise, with nothing raising anywhere. The backend dequantizes, and a test pins it. exllamav3 is written against its embedding-input path and not yet run. llama.cpp raises with the reason: the C API has both halves (llama_batch.embd, llama_get_embeddings) but llama-cpp-python does not surface them together, so it is a binding to write rather than a property of the format. 50/50 pass.
I said llama.cpp was the hard one and might be a wall. It was neither. The high-level Llama.eval() takes tokens only, which is why the lane looked closed, but llama-cpp-python exposes the whole C API underneath and that has both halves a brain needs: llama_batch_init(n_tokens, embd, n_seq) a batch whose `embd` field carries EMBEDDINGS, not ids llama_get_embeddings_ith(ctx, i) the final hidden state, per token Verified against a stock Q4_K_M build of Qwen2.5-0.5B: backend gguf, hidden 896 embed (1, 6, 896) forward logits (1, 6, 151936) hidden (1, 6, 896) hidden std 10.3633 That last number is the test that matters. llama.cpp's DEFAULT pooling returns one vector for the whole sequence, and a brain addressing on a pooled mean has nothing per-token to key on -- every write lands in the same place and the memory is uniformly useless without anything failing. The context is opened with pooling NONE, and a test pins it. What llama.cpp will not hand over is its output embedding matrix, and the brain's token codes come from that. Rather than guess, out_weight() raises and says the two ways through: train the brain against the transformers copy of the same model (token codes are a property of the vocabulary, not of the quantization) or use --codec bytes, which does not need the matrix at all. The byte payload was built two commits ago for tokenizer portability; it turns out to be what makes this lane self-sufficient too. exllamav3 is rewritten against its real module walk -- forward_ls() iterates fwd_modules, so skipping the first lets embeddings in and stopping before the last lets the hidden state out, no fork needed. Written, not yet run: there is no EXL3 model on the box to run it against. 51/51 pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…ld, verify it
Five things a user hits, and the measurement tool that would have caught them.
pollard-brainverify. The one measurement, with the construction that produced the published numbers built in. Rebuilding it by hand cost an evening and three different wrong answers -- 45.8%, 33.3%, 56.2% -- on a brain that measures 100%. It reports the floor, the recall, and BOTH leakage controls, and exits non-zero if any control is above 2%, because a number without its controls is not a measurement.
The query default was wrong, and so was my first correction. A token is filed under the words immediately before it, so retrieval reproduces that context. The verified prompt is "Question: what is the secret word? Answer: The secret word is" -- question AND continuation. Shipping only the question returns confident noise from a perfect memory; shipping only the continuation is wrong the same way. Both halves now, and a test pins it against the trainer's own ASKS and the verifier's construction.
recall() crashed on most documents. A final window shorter than
ekleft a short carry, and the next address slice died on a shape mismatch -- on any document whose length does not divide the window, which is nearly all of them.--continue-from. Training could only start over. On the same backbone this is cumulative curriculum -- run 900 steps, look at it, run 900 more. On a DIFFERENT backbone the memory-shaped parts (connectome, slots, codebook, decay) carry over and only the hidden-size-shaped layers are re-initialised, so a brain MOVES to a new model instead of starting from nothing; what is re-init reconverges in tens of steps because the memory it is learning to address is already organised. Changing bits/span between runs is refused outright -- that silently re-rolls the codebook and everything already written decodes to noise.
--codec bytes. The payload was this backbone's vocabulary indices, so only that tokenizer could read a brain back. Bytes store TEXT: any tokenizer can read it, and a six-letter word costs 6x8=48 bits against 4x18=72. The codec is recorded in the checkpoint and defaults to the original behaviour, because reading a brain with the wrong codec is silent garbage.
pollard --brain. A brain now ships WITH a build on every lane, with a BRAIN.md saying what it attaches to. GPTQ loads under transformers so it attaches at run time; GGUF, MLX and EXL3 run under engines with no hook for injecting memory tokens, so the file travels with the build and the note says so rather than leaving someone to discover it.
Also: training progress was invisible when redirected (block-buffered stdout, so
--train > logwrote nothing and read as a hung job), and the SKILL state-size line was stale at 8.8 MB after the width default moved to 328.48/48 pass.