Release 0.6.7: prepare the default DFlash2 engine at startup

.github/workflows/build_runpod.yml

@@ -9,7 +9,7 @@ on:
         default: "main"
         type: string
       deploy_tag:
-        description: "Deployment tag: 0.6.6 for this release (empty publishes dev tag only)"
+        description: "Deployment tag: 0.6.7 for this release (empty publishes dev tag only)"
         required: false
         default: ""
         type: string

CHANGELOG.md

@@ -2,6 +2,22 @@
 
 This is just a centralised version of the Github automatically generated Release changelogs.
 
+## 0.6.7
+
+* Align server startup preparation with the workspace default. A compatible configured DFlash2 drafter and its precision are selected before preload/warmup, so the first default Chat request reuses the prepared engine instead of unloading MTP and compiling a second verifier ([#501](https://github.com/eole-nlp/eole/pull/501)).
+* Preserve native startup settings when the drafter is missing or incompatible, validate token/ngram constraints before enabling DFlash2, and retain explicit later Off/MTP/precision switches ([#501](https://github.com/eole-nlp/eole/pull/501)).
+
+With `preload: true` and `warmup: true`, preparation finishes before server
+readiness. Later mode/precision changes or new input shapes may still require
+additional preparation. This correction needs a rebuilt full Eole image; updating
+only the RunPod wrapper does not apply it.
+
+Build both images from the **same `0.6.7` release commit**, validate on a GPU,
+then update the public RunPod template to `eole-runpod:0.6.7`. See the
+[publishing checklist](recipes/runpod/MAINTAINERS.md#publish-067).
+
+**Full Changelog**: https://github.com/eole-nlp/eole/compare/0.6.6...0.6.7
+
 ## 0.6.6
 
 * Add DFlash2 speculative decoding for Qwen3.8-27B, including a fixed-cache drafter CUDA graph, packed projections and optional INT8 drafter quantization ([#497](https://github.com/eole-nlp/eole/pull/497)).
@@ -16,7 +32,7 @@ benchmark rates depend on workload, settings and hardware.
 Publish both Docker images from the **same `0.6.6` release commit**, validate on
 a GPU, then update the existing public RunPod template to `eole-runpod:0.6.6`.
 Publishing the Python release does not build images or update that live template.
-See the [publishing checklist](recipes/runpod/MAINTAINERS.md#publish-066).
+See the [publishing checklist](recipes/runpod/MAINTAINERS.md).
 
 **Full Changelog**: https://github.com/eole-nlp/eole/compare/0.6.5...0.6.6
 

README.md

@@ -53,9 +53,14 @@ compilation waits are omitted; the displayed generation measurements come from
 Eole's inference worker on an RTX 5090. Benchmark numbers measure post-first-token
 decode speed and depend on the prompt, settings and hardware.
 
-## Latest: 0.6.6
+## Latest: 0.6.7
 
-This release adds DFlash2 decoding for Qwen3.8-27B and workspace controls for
+This patch aligns server startup warmup with the workspace default: when available,
+DFlash2/INT8 is prepared before readiness, avoiding a reload and compilation on
+the first default Chat request. Missing/incompatible drafters retain native startup
+settings.
+
+The workspace includes DFlash2 decoding for Qwen3.8-27B and controls for
 Off/MTP/DFlash2 with separate drafter and vocabulary-head precision. Generation
 settings open by default, and Chat allows up to 16,384 output tokens. The expanded
 benchmark compares code, English and translation across six decoding profiles,
@@ -65,14 +70,14 @@ and verifies the pinned drafter alongside the pre-converted showcase models.
 Eole supports **PyTorch >= 2.14.1 and < 2.15**. The refreshed
 Docker defaults use PyTorch 2.14.1 with CUDA 13.2. Start with the
 [workspace UI](recipes/ui/README.md), [Qwen3.8/MTP recipe](recipes/qwen38/README.md),
-or [Docker guide](docs/docker.md). See the [changelog](CHANGELOG.md#066) for release
+or [Docker guide](docs/docker.md). See the [changelog](CHANGELOG.md#067) for release
 details and remaining limitations.
 
 [![Deploy on RunPod](https://img.shields.io/badge/RunPod-Deploy%20Eole-673AB7?style=for-the-badge)](https://console.runpod.io/deploy?template=e9e19ozt9c&type=gpu)
 
 Browser-based Chat, translation and OCR on a rented GPU. See the
 [RunPod quick start](recipes/runpod/README.md) for password setup, GPU choices and
-the required persistent-volume attachment. The `0.6.6` deployment
+the required persistent-volume attachment. The `0.6.7` deployment
 uses prebuilt GPU kernels and pre-converted public Eole checkpoints. Maintainers
 publish the new images and update the public template before the button selects
 this release.

docs/docker.md

@@ -8,13 +8,13 @@ Configure the NVIDIA Docker runtime before starting a GPU container.
 
 Use the **full Eole image** for local testing. You do not need to build a base
 image, publish anything, or use the RunPod image. The commands below use the
-0.6.6 release image (after publication) and an already converted Qwen checkpoint.
+0.6.7 release image (after publication) and an already converted Qwen checkpoint.
 Until the image is published, build your checkout locally using the next section.
 
 Run these commands in a terminal on your host:
 
 ```bash
-export EOLE_IMAGE=ghcr.io/eole-nlp/eole:0.6.6-torch2.14.1-ubuntu24.04-cuda13.2.1
+export EOLE_IMAGE=ghcr.io/eole-nlp/eole:0.6.7-torch2.14.1-ubuntu24.04-cuda13.2.1
 export EOLE_MODEL_DIR=/mnt/InternalCrucial4/LLM_work
 
 # Check that Docker can access your GPUs. The first run downloads the image.
@@ -41,7 +41,7 @@ hf download incoai/Qwen3.8-27B-DFlash2 \
 The result must contain `dflash2/config.json` and `dflash2/model.safetensors`.
 The model-directory mount below makes them available inside Docker at the same
 path. No drafter conversion is needed. Without these files, DFlash2 stays
-disabled; Off and native MTP remain available. The 0.6.6 image includes the
+disabled; Off and native MTP remain available. The 0.6.7 image includes the
 DFlash2 workspace selector; older 0.6.5 images do not.
 
 Stop other inference processes using the selected GPU to free VRAM, then start
@@ -78,7 +78,7 @@ Press **Ctrl+C** to stop, or run `docker stop eole-local` in another terminal.
 save any other files you want to keep in a mounted directory.
 
 This runs the code packaged in the selected image. Mounting model files does
-not update that code. Use the 0.6.6 release image for the DFlash2 selector and
+not update that code. Use the 0.6.7 release image for the DFlash2 selector and
 workload/profile benchmark comparisons; older images do not gain these features.
 See the next section to build a local image from your checkout. For the combined
 Chat/Translate/OCR showcase, follow the [UI recipe](../recipes/ui/README.md) to
@@ -256,12 +256,12 @@ No unrelated release number is assigned to current main. Publishing requires
 GHCR package write permission; the workflow uses its repository `GITHUB_TOKEN`.
 **Build Base Docker Image** publishes just the base when needed.
 
-For the **0.6.6 release image**, after the version-bump PR is merged and the
-`0.6.6` tag is created on the release commit, select `source_ref=0.6.6` (keep the
+For the **0.6.7 release image**, after the version-bump PR is merged and the
+`0.6.7` tag is created on the release commit, select `source_ref=0.6.7` (keep the
 workflow branch on `main`). The workflow then publishes:
 
 ```text
-ghcr.io/eole-nlp/eole:0.6.6-torch2.14.1-ubuntu24.04-cuda13.2.1
+ghcr.io/eole-nlp/eole:0.6.7-torch2.14.1-ubuntu24.04-cuda13.2.1
 ```
 
 The release tag must include the refreshed Docker tooling. Publishing a GitHub
@@ -285,7 +285,7 @@ hardware before claiming a new combination is supported.
 
 ### RunPod deployment images
 
-Build the full Eole image from the **`0.6.6` tag** first. The image includes the
+Build the full Eole image from the **`0.6.7` tag** first. The image includes the
 current source, bundled CUDA ops, Torch 2.14.1, Triton 3.8.0 and fla-core 0.6.0.
 The full-image workflow builds its matching base too; a separate base workflow
 run is unnecessary. Its **Summary** shows the full image tag for the next step.
@@ -294,9 +294,9 @@ Run **Build RunPod Image** with:
 
 | Input | Value |
 |---|---|
-| `source_ref` | `0.6.6` |
-| `deploy_tag` | `0.6.6` |
-| `eole_image` | `ghcr.io/eole-nlp/eole:0.6.6-torch2.14.1-ubuntu24.04-cuda13.2.1` or its digest |
+| `source_ref` | `0.6.7` |
+| `deploy_tag` | `0.6.7` |
+| `eole_image` | `ghcr.io/eole-nlp/eole:0.6.7-torch2.14.1-ubuntu24.04-cuda13.2.1` or its digest |
 
 The wrapper requires importable bundled ops and startup/cache support. A plain
 release tag additionally requires a parent with the same Eole version and source
@@ -305,9 +305,9 @@ only the wrapper is rebuilt. Both builds install matching Torchvision while
 constraining the selected PyTorch/CUDA runtime.
 
 After GPU validation, update the existing public template's `imageName` to
-**`ghcr.io/eole-nlp/eole-runpod:0.6.6`**, following the
-[release publishing checklist](../recipes/runpod/MAINTAINERS.md#publish-066).
-The repository template targets 0.6.6, but Git merges and image publication do not
+**`ghcr.io/eole-nlp/eole-runpod:0.6.7`**, following the
+[release publishing checklist](../recipes/runpod/MAINTAINERS.md#publish-067).
+The repository template targets 0.6.7, but Git merges and image publication do not
 change the live template or existing Pods. The deploy button keeps its current
 template ID. Preserve earlier release/revision images rather than overwriting them.
 The launcher downloads pinned pre-converted public `eole-nlp` checkpoints and

eole/__init__.py

@@ -1,6 +1,6 @@
 import os
 
-__version__ = "0.6.6"
+__version__ = "0.6.7"
 ROOT_DIR = os.path.abspath(os.path.dirname(__file__))
 EOLE_TORCH_COMPILE = os.environ.get("EOLE_TORCH_COMPILE", "0") == "1"
 

eole/tests/test_runpod.py

@@ -511,7 +511,7 @@ def test_catalog_pins_all_models_and_release_template_packages_them(self):
             self.assertGreater(model["size_bytes"], 1024**3)
         recipe = Path(__file__).resolve().parents[2] / "recipes/runpod"
         template = json.loads((recipe / "template.json").read_text())
-        self.assertEqual(template["imageName"], "ghcr.io/eole-nlp/eole-runpod:0.6.6")
+        self.assertEqual(template["imageName"], "ghcr.io/eole-nlp/eole-runpod:0.6.7")
         self.assertIn('data-stage="verify"', (recipe / "preparing.html").read_text())
         self.assertNotIn('data-stage="convert"', (recipe / "preparing.html").read_text())
 

eole/tests/test_runpod_release.py

@@ -18,11 +18,11 @@ def setUp(self):
         self.script = recipe / "check_release.sh"
         shutil.copy2(Path(__file__).resolve().parents[2] / "recipes/runpod/check_release.sh", self.script)
         (self.root / "eole").mkdir()
-        (self.root / "eole/__init__.py").write_text('__version__ = "0.6.6"\n')
+        (self.root / "eole/__init__.py").write_text('__version__ = "0.6.7"\n')
         self.git("init", "-q")
         self.git("add", ".")
         self.git("-c", "user.name=Test", "-c", "user.email=test@example.com", "commit", "-qm", "release")
-        self.git("tag", "0.6.6")
+        self.git("tag", "0.6.7")
         revision = self.git("rev-parse", "HEAD").stdout.strip()
         bin_dir = self.root / "bin"
         bin_dir.mkdir()
@@ -41,14 +41,14 @@ def setUp(self):
             **os.environ,
             "PATH": f"{bin_dir}:{os.environ['PATH']}",
             "DOCKER_TEST_LOG": str(self.log),
-            "PARENT_VERSION": "0.6.6",
+            "PARENT_VERSION": "0.6.7",
             "PARENT_REVISION": revision,
         }
 
     def git(self, *args):
         return subprocess.run(["git", *args], cwd=self.root, text=True, capture_output=True, check=True)
 
-    def check(self, tag="0.6.6"):
+    def check(self, tag="0.6.7"):
         return subprocess.run(
             ["bash", str(self.script), tag, "example/eole:release"],
             cwd="/tmp",
@@ -63,7 +63,7 @@ def test_matching_source_and_parent_are_accepted(self):
         self.assertIn("image inspect", self.log.read_text())
 
     def test_stale_parent_version_or_revision_is_rejected(self):
-        for key, value in [("PARENT_VERSION", "0.6.5"), ("PARENT_REVISION", "0" * 40)]:
+        for key, value in [("PARENT_VERSION", "0.6.6"), ("PARENT_REVISION", "0" * 40)]:
             with self.subTest(key=key):
                 original = self.env[key]
                 self.env[key] = value
@@ -76,7 +76,7 @@ def test_untagged_source_is_rejected_before_pulling_images(self):
         self.git("-c", "user.name=Test", "-c", "user.email=test@example.com", "commit", "--allow-empty", "-qm", "later")
         result = self.check()
         self.assertNotEqual(result.returncode, 0)
-        self.assertIn("source_ref=0.6.6", result.stderr)
+        self.assertIn("source_ref=0.6.7", result.stderr)
         self.assertFalse(self.log.exists())
 
     def test_wrong_source_package_version_is_rejected(self):
@@ -86,7 +86,7 @@ def test_wrong_source_package_version_is_rejected(self):
         self.assertFalse(self.log.exists())
 
     def test_development_build_does_not_require_a_release_tag_or_pull(self):
-        self.git("tag", "-d", "0.6.6")
+        self.git("tag", "-d", "0.6.7")
         for tag in ("", "dev-123456789abc"):
             result = self.check(tag)
             self.assertEqual(result.returncode, 0, result.stderr)

recipes/runpod/MAINTAINERS.md

@@ -4,20 +4,20 @@ For deploying an existing template, follow the [user quick start](README.md).
 This guide is for maintainers publishing the images and deploy button, or users
 customizing the deployment. Ordinary users do not need Docker commands or an API key.
 
-## Publish 0.6.6
+## Publish 0.6.7
 
 The Python package, full Eole image and RunPod wrapper must include the merged
-0.6.6 source. Publishing a GitHub release does not build Docker images, and
+0.6.7 source. Publishing a GitHub release does not build Docker images, and
 publishing images does not update the live RunPod template.
 
-1. Merge the release PR, create tag **`0.6.6`** on that commit, and publish its
+1. Merge the release PR, create tag **`0.6.7`** on that commit, and publish its
    GitHub release. **Deploy Docs & Publish to PyPi** publishes the Python package
    and documentation. It checks that the tag matches the package version.
 2. Run **Actions → Build Docker Images** with the workflow branch on `main`:
 
    | Input | Value |
    |---|---|
-   | `source_ref` | `0.6.6` |
+   | `source_ref` | `0.6.7` |
    | `torch_version` | `2.14.1` |
    | `cuda_version` | `13.2.1` |
    | `cuda_arch_list` | `8.0 8.6 8.9 9.0 10.0 10.3 12.0` |
@@ -28,7 +28,7 @@ publishing images does not update the live RunPod template.
    image reference to use in the next step:
 
    ```text
-   ghcr.io/eole-nlp/eole:0.6.6-torch2.14.1-ubuntu24.04-cuda13.2.1
+   ghcr.io/eole-nlp/eole:0.6.7-torch2.14.1-ubuntu24.04-cuda13.2.1
    ```
 
    The default stack includes FlashAttention 2.8.3, Triton 3.8.0 and fla-core
@@ -38,11 +38,11 @@ publishing images does not update the live RunPod template.
 
    | Input | Value |
    |---|---|
-   | `source_ref` | `0.6.6` |
-   | `deploy_tag` | `0.6.6` |
+   | `source_ref` | `0.6.7` |
+   | `deploy_tag` | `0.6.7` |
    | `eole_image` | Full image reference above, or its immutable digest |
 
-   This publishes **`ghcr.io/eole-nlp/eole-runpod:0.6.6`**. The release check
+   This publishes **`ghcr.io/eole-nlp/eole-runpod:0.6.7`**. The release check
    rejects a parent whose Eole version or source revision differs from the tagged
    checkout. An older r2 image cannot supply the latest kernels, quantized heads,
    workspace controls or warmup code merely by rebuilding the wrapper.
@@ -53,9 +53,9 @@ publishing images does not update the live RunPod template.
    unloading and a cached restart. Build-time imports do not validate GPU execution.
 5. After validation, update the **existing public RunPod template `e9e19ozt9c`**
    in the console. Set **Container Image** to
-   **`ghcr.io/eole-nlp/eole-runpod:0.6.6`**; retain the launcher, Secret reference,
+   **`ghcr.io/eole-nlp/eole-runpod:0.6.7`**; retain the launcher, Secret reference,
    port and storage settings from `template.json`. Its repository copy already
-   selects 0.6.6; merging it does not update the live template.
+   selects 0.6.7; merging it does not update the live template.
 6. Open the README deploy button and check the selected image before deployment.
    The button's template ID stays unchanged. Existing Pods retain their images;
    stop/delete validation resources according to the agreed retention policy.
@@ -67,7 +67,7 @@ or CUDA code require rebuilding the full parent. For later Docker-only correctio
 use a new revision tag instead of overwriting published images. The strict
 same-release check applies to plain `x.y.z` deployment tags.
 
-Do not overwrite `0.6.4`, `0.6.4-r1`, `0.6.4-r2`, or a published `0.6.5` or `0.6.6` image.
+Do not overwrite `0.6.4`, `0.6.4-r1`, `0.6.4-r2`, or a published `0.6.5`, `0.6.6`, or `0.6.7` image.
 A deployment-only correction does not require another PyPI version. Building an
 image does not rent a Pod. See the [Docker guide](../../docs/docker.md) for local
 builds and compatibility requirements.
@@ -340,7 +340,7 @@ were terminated after validation; no test storage is retained.
 
 ## Startup preparation in the release image
 
-The 0.6.6 full Eole image contains the server, local verifier cache and timing
+The 0.6.7 full Eole image contains the server, local verifier cache and timing
 fixes. Its RunPod wrapper checks for importable ops and startup/cache support.
 The default Qwen model uses `preload: true, warmup: true`; the preparation screen
 stays visible until inference preparation finishes. Translation/OCR load on first

recipes/runpod/README.md

@@ -12,11 +12,11 @@ that volume; it does not attach it automatically. The 60 GB container disk alone
 is not sufficient. Private cloud validation has passed INT4/BF16 Chat,
 translation, PDF OCR and automatic model switching with an attached volume.
 
-This recipe targets **`eole-runpod:0.6.6`**, with prebuilt Eole CUDA ops and
+This recipe targets **`eole-runpod:0.6.7`**, with prebuilt Eole CUDA ops and
 pre-converted models from the public [eole-nlp organization](https://huggingface.co/eole-nlp).
 No HF token is required for these four checkpoints; no model conversion runs on the Pod.
 Maintainers must first build a full Eole image containing those ops, publish the
-RunPod wrapper, and update the existing public template before the button uses it; see the [publishing steps](MAINTAINERS.md#publish-066).
+RunPod wrapper, and update the existing public template before the button uses it; see the [publishing steps](MAINTAINERS.md#publish-067).
 
 ## Get started
 
@@ -97,7 +97,7 @@ and non-MTP paths retain their existing policy. New prompt/image shapes can stil
 need additional kernel warmup.
 
 **Publishing:** these startup changes require a full Eole image and RunPod wrapper
-built from this revision. Publishing the `0.6.6` image and updating the public template are separate steps;
+built from this revision. Publishing the `0.6.7` image and updating the public template are separate steps;
 merging this recipe leaves existing images and Pods unchanged. Update the template
 using [MAINTAINERS.md](MAINTAINERS.md). The wrapper build rejects parent images
 that lack the matching server/cache implementation.
@@ -171,4 +171,4 @@ of compiling it on the rented GPU. A source-build fallback handles incompatible
 binaries. Model downloads, checksum verification and Triton/torch.compile warmup still take time.
 Older published images retain their original startup behavior. Maintainers must
 publish a new image and update the deployment template to enable this change;
-see [publishing instructions](MAINTAINERS.md#publish-066).
+see [publishing instructions](MAINTAINERS.md#publish-067).

recipes/runpod/template.json

@@ -1,7 +1,7 @@
 {
   "name": "Eole Workspace",
   "category": "NVIDIA",
-  "imageName": "ghcr.io/eole-nlp/eole-runpod:0.6.6",
+  "imageName": "ghcr.io/eole-nlp/eole-runpod:0.6.7",
   "isPublic": false,
   "isServerless": false,
   "containerDiskInGb": 60,

setup.py

@@ -90,7 +90,7 @@ def get_ext_modules_and_cmdclass():
     description="Open language modeling toolkit based on PyTorch",
     long_description=long_description,
     long_description_content_type="text/markdown",
-    version="0.6.6",
+    version="0.6.7",
     packages=find_packages(),
     license_files=["LICENSE.md", "licenses/dflash-MIT.txt"],
     package_data={"eole.server": ["web/*"]},