Skip to content

Infer the preview variant from the mflux model - #43

Merged
IonDen merged 3 commits into
mainfrom
feat/variant-auto-inference
Sep 18, 2026
Merged

IonDen merged 3 commits into
mainfrom
feat/variant-auto-inference

Conversation

@IonDen

@IonDen IonDen commented Sep 18, 2026 •

Copy link
Copy Markdown
Owner

LivePreviewCallback(flux=model) now picks the tiny decoder from the model itself. Until now the quickstart needed variant="taef2" (or "taef1", "zimage", "qwen-image", "krea2") spelled out by hand, and the default was taef2, so a Z-Image or Qwen-Image user who copied the README and forgot the argument got a wrong-channel error at the first preview step, or one warning and no previews at all with on_error="disable". Resolution and batch-norm statistics were already read from the model; the variant was the last argument the user had to know.

Every mflux model object carries a model_config with model_name, base_model and its CLI aliases. Each preview kernel's mflux binding now lists the owner-qualified name prefix of the family it serves (black-forest-labs/FLUX.1-, black-forest-labs/FLUX.2-klein, Tongyi-MAI/Z-Image, Qwen/Qwen-Image, krea/Krea-2-) and the alias list mflux 0.19.1 registers for it. A resolver in mlx_taef.kernels tries three arms in order and stops at the first that claims the model: base_model (the canonical name behind a pinned mirror or a pre-quantized copy) against the prefixes, then model_name against the prefixes, then the aliases; every comparison folds case. When variant= is not given and flux= is, the callback uses it; an explicit variant= still wins; LivePreviewCallback() with no model keeps the taef2 default for pure-decoder use. A model outside the supported families (Lens, FIBO, ERNIE, Ideogram, Boogu, SeedVR2) raises UnsupportedMfluxModelError at construction, before any weights load, naming the model, the supported families and the override; the class is exported from the package root. The chosen variant is readable as callback.variant.

Two behaviours change for callers who passed flux=model without variant=. On a FLUX.2 Klein model nothing changes. On any other supported model they used to get TAEF2 and a wrong-channel error at the first preview step (or one warning and no previews under the default on_error="disable"); they now get the right decoder. An object without a usable model_config, or a model outside the supported families, used to run TAEF2 and now raises at construction; variant= keeps the old choice. The CHANGELOG states both.

README, EXAMPLES, the manual-verification guide and the three example scripts shrink to the one-liner, and the EXAMPLES snippets now construct the model they pass. Nothing changes for callers who pass variant= today. The FLUX.1 family maps Kontext, Fill, Depth, Redux, Krea-dev and the ControlNet entries to taef1: each hands the in-loop callback the same packed FLUX.1 latent, with conditioning carried separately. Krea 2 Raw maps to krea2 alongside Krea 2 Turbo because mflux runs both through one initializer, VAE and latent creator, and its weight definition shares the VAE between the two.

Evidence: 49 new offline tests. An exhaustive test walks every key of the installed mflux registry (34 in 0.19.1) against a complete expected map and asserts the key sets are equal, so a model mflux adds, renames or drops fails CI by name until someone classifies it; each entry is resolved through mflux's real ModelConfig objects, every alias mflux registers for a supported model is resolved on its own through the alias arm, and every alias a kernel lists must be one mflux registers. A hand-written table pins the same outcomes without mflux installed. Further tests cover the mirror-with-base_model case, alias-only local paths, case folding on names and on aliases, a family prefix appearing mid-name (must not match), a cross-family custom checkpoint (the base model wins), a bare string passed as aliases, the error message, an object without model_name, two kernels claiming one model, and alias/prefix disjointness plus lowercase aliases across the shipped kernels. At the callback level: inference picks Z-Image, an explicit variant wins, the no-model default holds, an unknown model raises before from_pretrained runs, the real Klein ModelConfig resolves to taef2, and a binding without a matching class reports the inferred name rather than None. Twenty-six one-line mutants (each of the above with the guard removed, the arms ORed instead of ordered, the Krea prefix narrowed, a registry key dropped from the map, an uppercase alias registered, the re-export removed, and others) were each killed by the named test. Full suite: 607 passed, coverage 98.7%.

LivePreviewCallback(flux=model) now resolves its tiny decoder from the
model's mflux config: each kernel binding names the family prefix it
previews and the full 0.19.1 alias list, and a resolver matches
base_model, then model_name, then aliases, requiring exactly one claim.
An explicit variant= still wins; no model keeps the taef2 default; a
model outside the supported families raises UnsupportedMfluxModelError
before any weights load, naming the override. The quickstart, EXAMPLES
and the example scripts drop the hand-picked variant argument.
@IonDen IonDen self-assigned this Sep 18, 2026
@IonDen IonDen added the enhancement New feature or request label Sep 18, 2026
The drift guard now iterates every key of the installed mflux registry
against a complete expected map and requires the key sets to match, so
a model mflux adds or renames fails CI by name. The resolver tries the
base model's prefix, then the model name's, then the aliases, stopping
at the first arm with a claimant, which gives a cross-family custom
checkpoint a clear answer. Krea 2 Raw maps to krea2: mflux runs it
through the same initializer, VAE and latent creator as Turbo. A bare
string passed as aliases is no longer split into characters, the
bad-variant error names the resolved value, UnsupportedMfluxModelError
is exported from the package root, the class docstring describes the
inferred default, the CHANGELOG states the behaviour change, and the
EXAMPLES snippets construct the model they pass.
The registry walk resolves mflux-built configs through the prefix arms,
so it could not see a kernel alias going stale or being dropped. Two
tests now cover the alias arm: every alias mflux registers for a
supported model resolves on its own with a name no prefix claims, and
every alias a kernel lists, outside the labelled legacy strings, is one
mflux registers. The alias arm no longer re-folds the registered side,
which the registry tests pin as lowercase.
@IonDen
IonDen merged commit ca196f2 into main Sep 18, 2026
7 checks passed
@IonDen
IonDen deleted the feat/variant-auto-inference branch September 18, 2026 21:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant