MFlux-Community Models

[ianscrivener]

https://huggingface.co/mflux-community
52 MFlux models in 11 Collections

[lpalbou]

Hi @ianscrivener , it's good to have models at different quants, however I currently maintained also several quants here : https://huggingface.co/AbstractFramework

One of the rationale is that a pure q4 or even sometime a q8 will degrade the rendering quality, so I spent quite some time creating mixed q4/q8/bf16 in order to preserve the quality of these models. Was anything like that done for the new mflux-community? Or are they just mlx quant? All of the models in my HF have been manually verified to render properly, do we have any similar guarantee for mflux-community?

[ianscrivener]

@lpalbou,

There are no mixed quants in MFlux yet... though I have put a PR in. The term "Mixed quants" can use can mean a few things so we need to unpack that.

Personally, I've had some good results with Text Encoder Q3, Diffusion Transformer Q6, VAE bf16. Just using the standard MFlux (MLX affine?) quantization method. "TE-q3, TR-q6, VAE-bf16".

This is the direction of my first PR for MFlux, so you will do;

    --model FLUX.2-klein-9b-kv-mflux-8bit \
    --model-text-encoder FLUX.2-klein-9b-kv-mflux-3bit \

Also had some good results with NVFP4. Perhaps not losing the dynamic range and keeping outliers adds to quality.

IMO It is counterproductive for quality to be quantizing the VAE. My code will be leaving it always as bf16.

As the bulk of the size and GPU time is in the transformer... my thought is to spend more time fine-tuning this. MLX has a few quant methods, there is also GGML style iMatrix. I also want to do some R&D on a rotate then quant technique (TBA).*

Prompt test set aware transformer quantizations are also a possibility (TBA).*

Also pruning and optimising the text-encoder. Eg ZIT & Krea: dropping Chinese, and optimizing and quantizing on say 10000 of my actual use prompts - reducing the vocab. Such methods have been used by https://huggingface.co/worstplayer/Z-Image_Qwen_3_4b_text_encoder_GGUF & BennyDaBall

* these may be novel - perhaps write 1 or 2 papers?

[ianscrivener]

I'll have to read your mlx-gen Quantization.md carefully to catch up on your mlx-gen quantizatrion recipes...