QwenImage21Cache device=cpu/auto leaves WDDM shared GPU memory mapped until process restart (Windows, Intel XPU) #16463

Activity

1. JWLHS commented on Oct 8, 2026

   

   JWLHS

   on Oct 8, 2026

   Author

   More actions

   Re-checked on the current tree and it still reproduces, plus we have narrowed the root cause down to a single platform condition. Adding the data here, plus the workaround we ship locally and what we would like to ask upstream for.

   Still current

   ComfyUI 0.39.0 (46493d824), comfy-aimdo 0.5.6, torch 2.13.0+xpu, Intel Arc A770 16GB, driver 32.0.101.8860, Windows 11. Qwen-Image 2.1 edit, 1024x1024, native int8 ConvRot UNETLoader, reference image (so the prefix/KV path is used), 12 steps, cfg 1.0. Per-process \GPU Process Memory(pid_*)\Shared Usage, same counter as this report:

   QwenImage21Cache device	peak during run	after the run
   off	17.8 MB	2.2 MB
   gpu	17.8 MB	2.2 MB
   cpu	1073 MB	1026 MB, never returns

   Fresh process baseline is ~0.4 MB. After the run RSS drops back to ~4.6 GB, so the host tensors themselves are released - only the mapping stays. POST /free {"unload_models": true, "free_memory": true} returns HTTP 200 and does not release it either; only exiting the process clears it.

   device=auto picks the device-side cache whenever there is spare VRAM and is clean; it only lands on this when VRAM is short, i.e. exactly the multi-reference / CLIP-pressure case from the original report.

   It is not the custom node, and not comfy-aimdo

   Both isolated, same workflow and same measurement:

   Configuration	peak shared GPU memory
   all custom-node patches off (OMNIXPU_ENABLE=0)	1793 MB
   runtime providers off (no comfy_aimdo.xpu, legacy ModelPatcher, native torch XPU allocator)	1665 MB
   custom node + aimdo enabled	1073 MB
   device=off in any of the above	17.8 MB

   So this is plain upstream ComfyUI + torch-xpu behaviour.

   Root cause

   QwenImage21Transformer2DModel.select_prefix_cache() builds PoseBranchCache(store_device=cpu) and put() copies the prefix K/V to the host. The only release path for that host side is:

   PoseBranchCache._free_slot() -> comfy.model_management.unpin_memory() -> torch.cuda.cudart().cudaHostUnregister()
   

   and in comfy/model_management.py MAX_PINNED_MEMORY is only assigned inside the is_nvidia() or is_amd() branch (line ~1637 in 0.39.0):

   if not DISABLE_PINNED_MEMORY:
       if is_nvidia() or is_amd():
           ... MAX_PINNED_MEMORY = ram * 0.40 ...

   On Intel XPU it stays -1, so pin_memory() returns False before touching the pointer, unpin_memory() returns immediately, and there is effectively no unregister step at all. The host pages end up mapped into the process' GPU address space by the driver and nothing ever asks for them back. On CUDA/ROCm the register/unregister pair is what makes that mapping explicit and revocable, which matches why this only shows up on XPU.

   Workaround we ship locally

   In our XPU custom node we gate the host store on the platform capability instead of the budget:

   • if MAX_PINNED_MEMORY <= 0 (host memory cannot be unregistered), refuse the host store and take the documented recompute fallback - device="cpu" then behaves like off, device="auto" still uses the device-side cache while VRAM is spare, device="gpu" is untouched, and CUDA/ROCm never enter the guard;
   • measured: device=cpu peak drops from 1073 MB to 2.4 MB, images are bit-identical to cache=off (mean_abs 0.000 / max 0.0), runtime 49.4s vs 51.6s before, and a 13-case regression suite stays green. device=gpu is also 2.4x faster than cpu here (20.2s vs 49.4s), so the VRAM cache is the better default on this card.

   The patch itself is small and lives in our adapter layer; PRs for reference: Blackwood416/ComfyUI-OmniXPU#11 and Blackwood416/llm-scaler#2.

   Request

   We would rather have this handled in core than carry the gate forever. Either direction would fix it:

   1. Make the host cache releasable - give PoseBranchCache's host tensors an allocation that can actually be unregistered/released on XPU (the same way pin_memory/unpin_memory do on CUDA/ROCm), so the mapping goes away in free(); or
   2. Refuse the host store when it cannot be released - treat "host memory is not registerable" as "no room for this slot" in select_prefix_cache() (and WanAnimate2Cache, which has the same PoseBranchCache on device="cpu" and would retain ~12.5 GB by its own node description at 480x832/81), and fall back to recompute. That is what we do locally and it is behaviourally transparent.

   If you tell us which shape you prefer and where it belongs, we are happy to send a PR against comfy/ldm/qwen_image21/model.py + comfy/model_management.py (option 2 is ~5 lines) instead of keeping it in an XPU adapter. Raw CSVs and the current probe logs are available if useful - just say the word.
2. JWLHS commented on Oct 9, 2026

   

   JWLHS

   on Oct 9, 2026

   Author

   More actions

   Adding the complete, self-contained reproduction recipe for this report, since the earlier comment only summarised it. Nothing custom is needed - stock ComfyUI nodes only, and the numbers are unchanged with every third-party patch disabled (OMNIXPU_ENABLE=0 gives 1793 MB instead of 1073 MB, still retained). Please try it and see whether the plateau comes back on your side.

   Environment

   Arc A770 16GB (DG2), Windows 11 build 26200, driver 32.0.101.8860, ComfyUI 0.39.0 (46493d824), torch 2.13.0+xpu (same on 2.14.0+xpu), comfy-aimdo 0.5.6, launched as python main.py --port 8188 --lowvram --disable-smart-memory --enable-dynamic-vram.

   Workflow (8 stock nodes, API format)

   UNETLoader -> QwenImage21Cache -> KSampler; TextEncodeQwenImage21 with one reference image -> prefix/KV cache path. Save as repro.json (adjust the model paths):

   {
     "1":  {"class_type": "UNETLoader", "inputs": {"unet_name": "qwen_image_2.1_int8_convrot.safetensors", "weight_dtype": "default"}},
     "3":  {"class_type": "CLIPLoader", "inputs": {"clip_name": "qwenimage2.1/qwen3vl_8b_int8_convrot.safetensors", "type": "qwen_image", "device": "default"}},
     "10": {"class_type": "VAELoader", "inputs": {"vae_name": "qwen/qwen_image_2.1_vae_bf16.safetensors"}},
     "20": {"class_type": "LoadImage", "inputs": {"image": "ref_test.jpg"}},
     "15": {"class_type": "QwenImage21Cache", "inputs": {"model": ["1", 0], "device": "cpu", "dtype": "default"}},
     "21": {"class_type": "TextEncodeQwenImage21", "inputs": {"clip": ["3", 0], "vae": ["10", 0], "images.image_1": ["20", 0], "resolution": 1024, "prompt": "turn her into a cyberpunk style portrait", "negative_prompt": "blurry, low quality"}},
     "6":  {"class_type": "EmptySD3LatentImage", "inputs": {"width": 1024, "height": 1024, "batch_size": 1}},
     "7":  {"class_type": "KSampler", "inputs": {"model": ["15", 0], "positive": ["21", 0], "negative": ["21", 1], "latent_image": ["6", 0], "seed": 4242, "steps": 12, "cfg": 1.0, "sampler_name": "euler", "scheduler": "simple", "denoise": 1.0}},
     "11": {"class_type": "VAEDecode", "inputs": {"samples": ["7", 0], "vae": ["10", 0]}},
     "12": {"class_type": "SaveImage", "inputs": {"images": ["11", 0], "filename_prefix": "repro/host_kv"}}
   }

   $body = '{"prompt": ' + (Get-Content .\repro.json -Raw) + '}'
   Invoke-RestMethod -Uri http://127.0.0.1:8188/prompt -Method Post -Body $body -ContentType 'application/json'

   Measuring shared GPU memory (same counter as Task Manager)

   $comfyPid = (Get-NetTCPConnection -State Listen -LocalPort 8188).OwningProcess
   function SharedMB {
     [math]::Round(((Get-Counter '\GPU Process Memory(*)\Shared Usage' |
       Select-Object -ExpandProperty CounterSamples |
       Where-Object { $_.InstanceName -match "^pid_${comfyPid}_" } |
       Measure-Object CookedValue -Sum).Sum / 1MB), 1)
   }
   "before     : $(SharedMB) MB"   # ~0.4 MB on a fresh process
   "after run  : $(SharedMB) MB"   # sample once "Prompt executed" is printed
   Start-Sleep 60
   "after +60s : $(SharedMB) MB"   # unchanged => the mapping is retained

   Results (only QwenImage21Cache.device changes between rows)

   device	peak during run	after the run	after +60 s
   off	17.8 MB	2.2 MB	2.2 MB
   gpu	17.8 MB	2.2 MB	2.2 MB
   cpu	1073 MB	1026 MB	1026 MB
   cpu, host stored in pinned memory (the XPU adapter in intel/llm-scaler does this)	2066 MB	2050 MB	2050 MB

   RSS falls back to ~4.6 GB after the run, so the host tensors are released - it is only the mapping that stays. POST /free {"unload_models": true, "free_memory": true} returns 200 and changes nothing; only exiting the process clears it.

   Why (as far as we can see)

   PoseBranchCache._free_slot() releases the host side through comfy.model_management.unpin_memory(), which calls torch.cuda.cudart().cudaHostUnregister(), and MAX_PINNED_MEMORY is only assigned for CUDA/ROCm. On XPU both are no-ops, so there is no unregister step at all and the driver keeps the mapped host pages.

   Two ways out that we can see: (1) give the host cache an allocation that can be released on XPU, or (2) when host memory cannot be unregistered, treat it as "no room for this slot" in select_prefix_cache() and fall back to recompute. We did (2) in the XPU custom node and the plateau disappears entirely (peak 2.4 MB, outputs bit-identical to device=off) - full write-up and PRs: intel/llm-scaler#752, Blackwood416/ComfyUI-OmniXPU#11.
3. JWLHS

   added 2 commits that reference this issue on Oct 9, 2026

   fix(adapters): refuse host KV cache when host memory cannot be unregi…

   ...

   cda0546

   omni/ComfyUI-OmniXPU: refuse host KV cache when host memory cannot be…

   ...

   520d2c8